[OpenWrt-Devel] Build issue using ccache due to GCC patch / R_MIPS_26
There seems to be an issue using ccache in combination with the openwrt patched gcc. Found the issue while investigating the following compilation issue: --- openwrt/staging_dir/toolchain-mips_gcc-4.6-linaro_uClibc-0.9.33.2/lib/gcc/mips-openwrt-linux-uclibc/4.6.3/../../../../mips-openwrt-linux-uclibc/bin/ld: .libs/libxt_ACCOUNT_cl.o: relocation R_MIPS_26 against `memset' can not be used when making a shared object; recompile with -fPIC libs/libxt_ACCOUNT_cl.o: could not read symbols: Bad value collect2: ld returned 1 exit status make[8]: *** [libxt_ACCOUNT_cl.la] Error 1 --- This only happened sometimes and only on certain machines. Closer look showed that libtool wasn't actually adding -fPIC. This was caused by the autoconf check for the PIC flag sometimes failing on my machine: checking for ccache_cc option to produce PIC... -fPIC -DPIC checking if ccache_cc PIC flag -fPIC -DPIC works... no Doing a 'clean prepare compile' of the package sometimes fixed it. The problem is that the autoconf test is doing a compilation with -fPIC then compares the stdout of the compiler to a 'normal' compilation. The results have to be equal to pass the test, in my setup, this wasn't always the case. The reason was that the compilation output for that step sometimes gave a warning: warning: someone does not honour COPTS correctly, passed 2 times This warning comes from the 910-mbsd_multi.patch patch to gcc, which verifies that -fhonour-copts is on the compilation line and that it is only present once. The patch also uses an environment variable called GCC_HONOUR_COPTS which indicates what has to happen if the option isn't present or is specified more than once. (possible values 0, 1, 2 and 's' for silent) The problem that was occuring on my setup was that -fhonour-copts had been specified more than once for an autoconf check and that this result was cached by ccache. All subsequent compilations for the same autoconf conftest.c file, returned the result from the compiler cache as for ccache the same inputs where used for the compilation. The problem here is that ccache has no knowledge that the environment variable GCC_HONOUR_COPTS influences the stdout from gcc and since the stdout is of importance to autoconf, it was causing the PIC config test to fail due to a prior cached compilation result. To verify my finding, I tried compilation without ccache, which worked and then tried patching ccache to also track GCC_HONOUR_COPTS, which also produced the correct result: --- ccache-3.1.7/ccache.c 2012-01-08 15:40:55.0 +0100 +++ ccache-3.1.7-patched/ccache.c 2015-07-14 09:56:02.037675777 +0200 @@ -965,6 +965,7 @@ calculate_object_hash(struct args *args, "CPLUS_INCLUDE_PATH", "OBJC_INCLUDE_PATH", "OBJCPLUS_INCLUDE_PATH", /* clang */ +"GCC_HONOUR_COPTS", NULL }; for (p = envvars; *p != NULL ; ++p) { With this patch, ccache will also add the GCC_HONOUR_COPTS setting to the hash of the compilation command and will consider 2 compilations with a different GCC_HONOUR_COPTS to actually be 2 different compilations. However, this doesn't completely fix the problem as it appears that the tools/ccache/Makefile will only be compiled if ccache isn't already present on the host system, so probably the proper way to fix this is to just unconditionally compile ccache too if it is selected by the build system and then add the above ccache patch to the package. Regards, Karl ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH] Make ccache aware of OpenWRT provided GCC patch.
The OpenWRT GCC patch, 910-mbsd_multi.patch adds an extra option to gcc that depends on an environment variable. Standard ccache is unaware of this option and therefor can produce stdout from gcc that doesn't correspond to what you would get if you actually do a direct compilation with gcc. This commit adds a patch to ccache to make it aware of the new option and removes the feature from the ccache package to use the host system's installed ccache binary. --- tools/ccache/Makefile | 24 -- tools/ccache/patches/100-honour-copts.patch | 31 + 2 files changed, 35 insertions(+), 20 deletions(-) create mode 100644 tools/ccache/patches/100-honour-copts.patch diff --git a/tools/ccache/Makefile b/tools/ccache/Makefile index 9020c9c..75c5f8c 100644 --- a/tools/ccache/Makefile +++ b/tools/ccache/Makefile @@ -24,29 +24,13 @@ define Host/Install/ccache $(CP) ./files/* $(STAGING_DIR_HOST)/bin/ endef -ifneq ($(strip $(shell which ccache >/dev/null && echo found)),found) - define Host/Clean +define Host/Clean -$(MAKE) -C $(HOST_BUILD_DIR) uninstall $(call Host/Clean/Default) - endef - define Host/Install +endef +define Host/Install $(call Host/Install/Default) $(call Host/Install/ccache) - endef -else - define Host/Prepare - endef - define Host/Configure - endef - define Host/Compile - endef - define Host/Install - $(call Host/Install/ccache) - endef - define Host/Clean - endef - define Download - endef -endif +endef $(eval $(call HostBuild)) diff --git a/tools/ccache/patches/100-honour-copts.patch b/tools/ccache/patches/100-honour-copts.patch new file mode 100644 index 000..9182d33 --- /dev/null +++ b/tools/ccache/patches/100-honour-copts.patch @@ -0,0 +1,31 @@ +From 135d868c539f994afbcf80d313ba2e39f031d3b1 Mon Sep 17 00:00:00 2001 +From: Karl Vogel +Date: Tue, 14 Jul 2015 11:05:33 +0200 +Subject: [PATCH] Include environment variable GCC_HONOUR_COPTS in hash. + +The OpenWRT patch, 910-mbsd_multi.patch, to GCC adds an extra +compilation flag, -fhonour-copts, which is influenced by an +environment variable called GCC_HONOUR_COPTS. + +Include this environment var in the hash calculation as otherwise +the gcc stdout warning from a previous compilation might be shown +where, even when GCC_HONOUR_COPTS is in 's'ilent mode. +--- + ccache.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/ccache.c b/ccache.c +index e41af13..fa67ed2 100644 +--- a/ccache.c b/ccache.c +@@ -965,6 +965,7 @@ calculate_object_hash(struct args *args, struct mdfour *hash, int direct_mode) + "CPLUS_INCLUDE_PATH", + "OBJC_INCLUDE_PATH", + "OBJCPLUS_INCLUDE_PATH", /* clang */ ++"GCC_HONOUR_COPTS", + NULL + }; + for (p = envvars; *p != NULL ; ++p) { +-- +1.9.1 + -- 1.9.1 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH] Make ccache aware of OpenWRT provided GCC patch.
The OpenWRT GCC patch, 910-mbsd_multi.patch adds an extra option to gcc that depends on an environment variable. Standard ccache is unaware of this option and therefor can produce stdout from gcc that doesn't correspond to what you would get if you actually do a direct compilation with gcc. This commit adds a patch to ccache to make it aware of the new option and removes the feature from the ccache package to use the host system's installed ccache binary. --- tools/ccache/Makefile | 24 -- tools/ccache/patches/100-honour-copts.patch | 31 + 2 files changed, 35 insertions(+), 20 deletions(-) create mode 100644 tools/ccache/patches/100-honour-copts.patch diff --git a/tools/ccache/Makefile b/tools/ccache/Makefile index 9020c9c..75c5f8c 100644 --- a/tools/ccache/Makefile +++ b/tools/ccache/Makefile @@ -24,29 +24,13 @@ define Host/Install/ccache $(CP) ./files/* $(STAGING_DIR_HOST)/bin/ endef -ifneq ($(strip $(shell which ccache >/dev/null && echo found)),found) - define Host/Clean +define Host/Clean -$(MAKE) -C $(HOST_BUILD_DIR) uninstall $(call Host/Clean/Default) - endef - define Host/Install +endef +define Host/Install $(call Host/Install/Default) $(call Host/Install/ccache) - endef -else - define Host/Prepare - endef - define Host/Configure - endef - define Host/Compile - endef - define Host/Install - $(call Host/Install/ccache) - endef - define Host/Clean - endef - define Download - endef -endif +endef $(eval $(call HostBuild)) diff --git a/tools/ccache/patches/100-honour-copts.patch b/tools/ccache/patches/100-honour-copts.patch new file mode 100644 index 000..9182d33 --- /dev/null +++ b/tools/ccache/patches/100-honour-copts.patch @@ -0,0 +1,31 @@ +From 135d868c539f994afbcf80d313ba2e39f031d3b1 Mon Sep 17 00:00:00 2001 +From: Karl Vogel +Date: Tue, 14 Jul 2015 11:05:33 +0200 +Subject: [PATCH] Include environment variable GCC_HONOUR_COPTS in hash. + +The OpenWRT patch, 910-mbsd_multi.patch, to GCC adds an extra +compilation flag, -fhonour-copts, which is influenced by an +environment variable called GCC_HONOUR_COPTS. + +Include this environment var in the hash calculation as otherwise +the gcc stdout warning from a previous compilation might be shown +where, even when GCC_HONOUR_COPTS is in 's'ilent mode. +--- + ccache.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/ccache.c b/ccache.c +index e41af13..fa67ed2 100644 +--- a/ccache.c b/ccache.c +@@ -965,6 +965,7 @@ calculate_object_hash(struct args *args, struct mdfour *hash, int direct_mode) + "CPLUS_INCLUDE_PATH", + "OBJC_INCLUDE_PATH", + "OBJCPLUS_INCLUDE_PATH", /* clang */ ++"GCC_HONOUR_COPTS", + NULL + }; + for (p = envvars; *p != NULL ; ++p) { +-- +1.9.1 + -- 1.9.1 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH] ccache: Make ccache aware of OpenWRT provided GCC patch.
The OpenWRT GCC patch, 910-mbsd_multi.patch adds an extra option to gcc that depends on an environment variable. Standard ccache is unaware of this option and therefor can produce stdout from gcc that doesn't correspond to what you would get if you actually do a direct compilation with gcc. This commit adds a patch to ccache to make it aware of the new option and removes the feature from the ccache package to use the host system's installed ccache binary. Signed-off-by: Karl Vogel --- tools/ccache/Makefile | 24 - tools/ccache/patches/100-honour-copts.patch | 33 + 2 files changed, 37 insertions(+), 20 deletions(-) create mode 100644 tools/ccache/patches/100-honour-copts.patch diff --git a/tools/ccache/Makefile b/tools/ccache/Makefile index 9020c9c..75c5f8c 100644 --- a/tools/ccache/Makefile +++ b/tools/ccache/Makefile @@ -24,29 +24,13 @@ define Host/Install/ccache $(CP) ./files/* $(STAGING_DIR_HOST)/bin/ endef -ifneq ($(strip $(shell which ccache >/dev/null && echo found)),found) - define Host/Clean +define Host/Clean -$(MAKE) -C $(HOST_BUILD_DIR) uninstall $(call Host/Clean/Default) - endef - define Host/Install +endef +define Host/Install $(call Host/Install/Default) $(call Host/Install/ccache) - endef -else - define Host/Prepare - endef - define Host/Configure - endef - define Host/Compile - endef - define Host/Install - $(call Host/Install/ccache) - endef - define Host/Clean - endef - define Download - endef -endif +endef $(eval $(call HostBuild)) diff --git a/tools/ccache/patches/100-honour-copts.patch b/tools/ccache/patches/100-honour-copts.patch new file mode 100644 index 000..5496dc1 --- /dev/null +++ b/tools/ccache/patches/100-honour-copts.patch @@ -0,0 +1,33 @@ +From e9a75b66ebb0b5b289b69a188b8b16306151bd11 Mon Sep 17 00:00:00 2001 +From: Karl Vogel +Date: Tue, 14 Jul 2015 11:05:33 +0200 +Subject: [PATCH] Include environment variable GCC_HONOUR_COPTS in hash. + +The OpenWRT patch, 910-mbsd_multi.patch, to GCC adds an extra +compilation flag, -fhonour-copts, which is influenced by an +environment variable called GCC_HONOUR_COPTS. + +Include this environment var in the hash calculation as otherwise +the gcc stdout warning from a previous compilation might be shown +where, even when GCC_HONOUR_COPTS is in 's'ilent mode. + +Signed-off-by: Karl Vogel +--- + ccache.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/ccache.c b/ccache.c +index e41af13..fa67ed2 100644 +--- a/ccache.c b/ccache.c +@@ -965,6 +965,7 @@ calculate_object_hash(struct args *args, struct mdfour *hash, int direct_mode) + "CPLUS_INCLUDE_PATH", + "OBJC_INCLUDE_PATH", + "OBJCPLUS_INCLUDE_PATH", /* clang */ ++"GCC_HONOUR_COPTS", + NULL + }; + for (p = envvars; *p != NULL ; ++p) { +-- +1.9.1 + -- 1.9.1 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH] ccache: Make ccache aware of OpenWRT provided GCC patch.
The OpenWRT GCC patch, 910-mbsd_multi.patch adds an extra option to gcc that depends on an environment variable. Standard ccache is unaware of this option and therefor can produce stdout from gcc that doesn't correspond to what you would get if you actually do a direct compilation with gcc. This commit adds a patch to ccache to make it aware of the new option and removes the feature from the ccache package to use the host system's installed ccache binary. Signed-off-by: Karl Vogel --- tools/ccache/Makefile | 24 - tools/ccache/patches/100-honour-copts.patch | 33 + 2 files changed, 37 insertions(+), 20 deletions(-) create mode 100644 tools/ccache/patches/100-honour-copts.patch diff --git a/tools/ccache/Makefile b/tools/ccache/Makefile index 9020c9c..75c5f8c 100644 --- a/tools/ccache/Makefile +++ b/tools/ccache/Makefile @@ -24,29 +24,13 @@ define Host/Install/ccache $(CP) ./files/* $(STAGING_DIR_HOST)/bin/ endef -ifneq ($(strip $(shell which ccache >/dev/null && echo found)),found) - define Host/Clean +define Host/Clean -$(MAKE) -C $(HOST_BUILD_DIR) uninstall $(call Host/Clean/Default) - endef - define Host/Install +endef +define Host/Install $(call Host/Install/Default) $(call Host/Install/ccache) - endef -else - define Host/Prepare - endef - define Host/Configure - endef - define Host/Compile - endef - define Host/Install - $(call Host/Install/ccache) - endef - define Host/Clean - endef - define Download - endef -endif +endef $(eval $(call HostBuild)) diff --git a/tools/ccache/patches/100-honour-copts.patch b/tools/ccache/patches/100-honour-copts.patch new file mode 100644 index 000..dc32885 --- /dev/null +++ b/tools/ccache/patches/100-honour-copts.patch @@ -0,0 +1,33 @@ +From 90762a9b8d9a50b6176f10bd6c2e2b9501117561 Mon Sep 17 00:00:00 2001 +From: Karl Vogel +Date: Tue, 14 Jul 2015 11:05:33 +0200 +Subject: [PATCH] Include environment variable GCC_HONOUR_COPTS in hash. + +The OpenWRT patch, 910-mbsd_multi.patch, to GCC adds an extra +compilation flag, -fhonour-copts, which is influenced by an +environment variable called GCC_HONOUR_COPTS. + +Include this environment var in the hash calculation as otherwise +the gcc stdout warning from a previous compilation might be shown +where, even when GCC_HONOUR_COPTS is in 's'ilent mode. + +Signed-off-by: Karl Vogel +--- + ccache.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/ccache.c b/ccache.c +index e41af13..b736a9c 100644 +--- a/ccache.c b/ccache.c +@@ -965,6 +965,7 @@ calculate_object_hash(struct args *args, struct mdfour *hash, int direct_mode) + "CPLUS_INCLUDE_PATH", + "OBJC_INCLUDE_PATH", + "OBJCPLUS_INCLUDE_PATH", /* clang */ ++ "GCC_HONOUR_COPTS", + NULL + }; + for (p = envvars; *p != NULL ; ++p) { +-- +1.9.1 + -- 1.9.1 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] fstools: MTD driver - deadc0de marker
I ran into an issue on a platform using a JFFS2 partition where the overlayfs mount was done with tmpfs. After some investigation the issue seemed to happen in the mount_root binary, more specifically during the mtd_volume_identify() function. The code does the check: if (v->type == UBIVOLUME && deadc0de == 0x) { return FS_JFFS2; } see http://git.openwrt.org/project/fstools.git?p=project/fstools.git;a=blob;f=libfstools/mtd.c;h=156d166b5a2cc92f124852180782e5c2826be881;hb=HEAD#l215 However, in my case the volume type is NANDFLASH. Adding a v->type == NANDFLASH in this place, makes my system work, but I'm not entirely sure why this deadc0de check was restricted to UBIVOLUME only?! Would a check for NANDFLASH be acceptable here? Regards, Karl ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] Read-only mirror https://github.com/openwrt/openwrt is stuck since 12 days ago
On Tue, Mar 15, 2016 at 6:22 PM, Bruno Randolf wrote: > On 03/07/2016 10:51 AM, John Crispin wrote: > > > > > > On 07/03/2016 11:47, Álvaro Fernández Rojas wrote: > >> I just want to add that openwrt-commits list is also affected, since > last commit published there matches the last one in the read-only mirror. > >> > >> Regards, > >> Álvaro. > > > > i noticed that one aswell, will investigate ... sorry for the > > delay/inconvenience > > It is still down: > > https://github.com/openwrt/openwrt? > "Updated 21 days ago"... > > Would be great to have the git repo synced to svn again. > > Thanks, > bruno > Doesn't look like the SVN server is back yet either. Are there any plans to resurrect the anon SVN server? Or should everybody move to git.openwrt.org instead? > >>> El 6 mar 2016, a las 19:33, Hannu Nyman escribió: > >>> > >>> The read-only mirror for the main Openwrt sources at Github has been > stuck since 12 days ago. > >>> > >>> https://github.com/openwrt/openwrt > >>> > >>> Both trunk and Chaos Calmer have got new commits since then... > >>> > >>> Apparently the mirror has not returned to automatic updates after the > network/hardware troubles. > ___ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel > ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH] toolchain/kernel-headers: use target config when generating headers
Without this change, the oldconfig could actually use the host system's kernel defaults from /boot, ie on an ubuntu build node the oldconfig would give: # using defaults found in /boot/config-3.13.0-77-generic This patch uses the Kernel/Prepare and Kernel/Configure for the actual target to generate the .config. While the kernel to userspace ABI is fixed, there might be additions to the kernel which would not be known when using the host configuration. ie. in the past the iptables Makefile was altered because the toolchain kernel headers were actually missing the layer7 headers (layer7 patches has since been dropped), but it indicates the issue. With this patch, the toolchain kernel headers will be those of the actual target kernel. Signed-off-by: Karl Vogel --- include/kernel-defaults.mk| 2 ++ toolchain/kernel-headers/Makefile | 7 +++ 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/include/kernel-defaults.mk b/include/kernel-defaults.mk index 406fd46..19ff994 100644 --- a/include/kernel-defaults.mk +++ b/include/kernel-defaults.mk @@ -66,6 +66,7 @@ else ln -s $(CONFIG_EXTERNAL_KERNEL_TREE) $(LINUX_DIR) endef endif +Kernel/Prepare?=$(Kernel/Prepare/Default) ifeq ($(CONFIG_TARGET_ROOTFS_INITRAMFS),y) ifeq ($(strip $(CONFIG_EXTERNAL_CPIO)),"") @@ -122,6 +123,7 @@ define Kernel/Configure/Default $(_SINGLE) [ -d $(LINUX_DIR)/user_headers ] || $(MAKE) $(KERNEL_MAKEOPTS) INSTALL_HDR_PATH=$(LINUX_DIR)/user_headers headers_install $(SH_FUNC) grep '=[ym]' $(LINUX_DIR)/.config.set | LC_ALL=C sort | md5s > $(LINUX_DIR)/.vermagic endef +Kernel/Configure?=$(Kernel/Configure/Default) define Kernel/Configure/Initramfs $(call Kernel/SetInitramfs) diff --git a/toolchain/kernel-headers/Makefile b/toolchain/kernel-headers/Makefile index 68c83a0..fae20d4 100644 --- a/toolchain/kernel-headers/Makefile +++ b/toolchain/kernel-headers/Makefile @@ -46,9 +46,7 @@ KMAKE := $(MAKE) -C $(HOST_BUILD_DIR) \ define Host/Configure/all mkdir -p $(BUILD_DIR_TOOLCHAIN)/linux-dev - $(KMAKE) \ - INSTALL_HDR_PATH="$(BUILD_DIR_TOOLCHAIN)/linux-dev/" \ - headers_install + $(CP) $(LINUX_DIR)/user_headers/* $(BUILD_DIR_TOOLCHAIN)/linux-dev/ endef # XXX: the following is needed to build lzma-loader @@ -71,12 +69,13 @@ define Host/Configure/post/mipsel endef define Host/Prepare - $(call Kernel/Prepare/Default) + $(call Kernel/Prepare) ln -sf linux-$(LINUX_VERSION) $(BUILD_DIR_TOOLCHAIN)/linux $(SED) 's/@expr length/@-expr length/' $(HOST_BUILD_DIR)/Makefile endef define Host/Configure + $(call Kernel/Configure) env yes '' | $(KMAKE) oldconfig $(call Host/Configure/all) -- 2.7.4 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH] ubus: Correct usage of timeout on poll function
As the man page explains: "Specifying a timeout of zero causes poll() to return immediately, even if no file descriptors are ready." The use of 0 as timeout could cause libubus to busy loop if the socket was non-blocking. For blocking sockets, this was less apparent as the subsequent read() would then block. Signed-off-by: Karl Vogel --- libubus-io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libubus-io.c b/libubus-io.c index 9d3ac6c..e6d4236 100644 --- a/libubus-io.c +++ b/libubus-io.c @@ -54,7 +54,7 @@ static void wait_data(int fd, bool write) struct pollfd pfd = { .fd = fd }; pfd.events = write ? POLLOUT : POLLIN; - poll(&pfd, 1, 0); + poll(&pfd, 1, -1); } static int writev_retry(int fd, struct iovec *iov, int iov_len, int sock_fd) @@ -321,7 +321,7 @@ void __hidden ubus_poll_data(struct ubus_context *ctx, int timeout) .events = POLLIN | POLLERR, }; - poll(&pfd, 1, timeout); + poll(&pfd, 1, timeout ? timeout : -1); ubus_handle_data(&ctx->sock, ULOOP_READ); } -- 2.6.3 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] Mismatch in ubus / uloop options
In ubus_reconnect, the ctx->sock.fd is created and set to non-blocking mode: fcntl(ctx->sock.fd, F_SETFL, fcntl(ctx->sock.fd, F_GETFL) | O_NONBLOCK | > O_CLOEXEC); > http://git.openwrt.org/?p=project/ubus.git;a=blob;f=libubus-io.c;h=9d3ac6c33f4d926c7f5c5d2033d531fc5fc6f834;hb=259450f414d8c9ee41896e8e6d6bc57ec00e2b63#l396 However, further in the call stack, the socket will get added to uloop via ubus_add_uloop with the flag ULOOP_BLOCKING set. http://git.openwrt.org/?p=project/ubus.git;a=blob;f=libubus.h;h=54e7d481b0496e719f2f567b664a2619c82a3b16;hb=259450f414d8c9ee41896e8e6d6bc57ec00e2b63#l245 ie when called from ubus_complete_request on an !registered ctx->sock, the ubus_add_uloop is done: http://git.openwrt.org/?p=project/ubus.git;a=blob;f=libubus-req.c;h=bf19f3670ea5b215190dfcce125c358f616c72a4;hb=259450f414d8c9ee41896e8e6d6bc57ec00e2b63#l136 The uloop function assumes a socket is blocking by default and will never make a socket blocking again, it will only turn it to non-blocking if ULOOP_BLOCKING isn't set. This caused a busy loop in some daemons that had their socket set to non-blocking due to another issue, ie. poll using 0 as infinite instead of -1, for which I submitted a fix earlier. I wonder if it wouldn't be better to have uloop set the socket flags in all cases? Currently it "assumes" the socket is blocking and if ULOOP_BLOCKING is passed, it will set it to non-blocking, but it won't do the other way round. Though that might have some impact on daemons that think the socket is non-blocking while it will be turned into blocking if this is changed. So not sure what the best way to go about this one is?! Regards, Karl ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] network/znc: Package binaries appear to be broken by MIPS16
On Tue, Jan 20, 2015 at 11:07 AM, Sławomir Demeszko wrote: > 19.01.2015 19:50, Oliver wrote: >> >> I'll try 4.9 - out of interest, do you have any references to the bug in >> 4.8 (C++ specific?) > > > Sorry, I do not remember where I read about that. I think you're referring to the MIPS16 bug in qemu: http://wiki.openwrt.org/doc/howto/qemu#openwrt_in_qemu_mips ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH][packages] Enable better customization of nginx features.
Add configuration settings to disable certain features in nginx to be able to reduce it's footprint and allow nginx to be built with support for lua. Signed-off-by: Karl Vogel --- net/nginx/Config.in | 131 + net/nginx/Makefile| 125 +++- net/nginx/patches-lua-nginx/300-ldl.patch | 23 + 3 files changed, 277 insertions(+), 2 deletions(-) create mode 100644 net/nginx/patches-lua-nginx/300-ldl.patch diff --git a/net/nginx/Config.in b/net/nginx/Config.in index e451037..1396d10 100644 --- a/net/nginx/Config.in +++ b/net/nginx/Config.in @@ -36,4 +36,135 @@ config NGINX_DAV help Enable the HTTP and WebDAV methods PUT, DELETE, MKCOL, COPY and MOVE. +config NGINX_MODULE_LUA + bool + prompt "Enable LUA module" + default n + help + Enable support for LUA scripts. + + menu "Disable features" + + config NGINX_DISABLE_PCRE + bool + prompt "Disable PCRE library usage" + default n + + config NGINX_DISABLE_HTTP_CACHE + depends on NGINX_SSL=y + bool + prompt "Disable HTTP cache" + default y + + config NGINX_DISABLE_HTTP_CHARSET_MODULE + bool + prompt "Disable HTTP charset module" + default n + + config NGINX_DISABLE_HTTP_GZIP_MODULE + bool + prompt "Disable HTTP gzip module" + default n + + config NGINX_DISABLE_HTTP_SSI_MODULE + bool + prompt "Disable HTTP ssi module" + default n + + config NGINX_DISABLE_HTTP_USERID_MODULE + bool + prompt "Disable HTTP userid module" + default n + + config NGINX_DISABLE_HTTP_ACCESS_MODULE + bool + prompt "Disable HTTP access module" + default n + + config NGINX_DISABLE_HTTP_AUTH_BASIC_MODULE + bool + prompt "Disable HTTP auth basic" + default n + + config NGINX_DISABLE_HTTP_AUTOINDEX_MODULE + bool + prompt "Disable HTTP autoindex module" + default n + + config NGINX_DISABLE_HTTP_GEO_MODULE + bool + prompt "Disable HTTP geo module" + default n + + config NGINX_DISABLE_HTTP_MAP_MODULE + bool + prompt "Disable HTTP map module" + default n + + config NGINX_DISABLE_HTTP_SPLIT_CLIENTS_MODULE + bool + prompt "Disable HTTP split clients" + default n + + config NGINX_DISABLE_HTTP_REFERER_MODULE + bool + prompt "Disable HTTP referer module" + default n + + config NGINX_DISABLE_HTTP_REWRITE_MODULE + bool + prompt "Disable HTTP rewrite module" + default n + + config NGINX_DISABLE_HTTP_PROXY_MODULE + bool + prompt "Disable HTTP proxy module" + default n + + config NGINX_DISABLE_HTTP_FASTCGI_MODULE + bool + prompt "Disable HTTP fastcgi module" + default n + + config NGINX_DISABLE_HTTP_UWSGI_MODULE + bool + prompt "Disable HTTP uwsgi module" + default n + + config NGINX_DISABLE_HTTP_SCGI_MODULE + bool + prompt "Disable HTTP scgi module" + default n + + config NGINX_DISABLE_HTTP_MEMCACHED_MODULE + bool + prompt "Disable HTTP memcached module" + default n + + config NGINX_DISABLE_HTTP_LIMIT_CONN_MODULE + bool + prompt "Disable HTTP limit conn" + default n + + config NGINX_DISABLE_HTTP_LIMIT_REQ_MODULE + bool + prompt "Disable HTTP limit req" + default n + + config NGINX_DISABLE_HTTP_EMPTY_GIF_MODULE + bool + prompt "Disable HTTP empty gif" + default n + + config NGINX_DISABLE_HTTP_BROWSER_MODULE + bool + prompt "Disable HTTP browser module" + default n + + config NGINX_DISABLE_HTTP_UPSTREAM_IP_HASH_MODULE + bool + prompt "Disable HTTP IP hash module" + default n + + endmenu endmenu diff --git a/net/nginx/Makefile b/net/nginx/Makefile index d56a729..e6fcbc8 100644 --- a/net/nginx/Makefile +++ b/net/nginx/Makefile @@ -22,7 +22,31 @@ PKG_CONFIG_DEPENDS := \ CONFIG_NGINX_STUB_STATUS \ CONFIG_NGINX_FLV \ CONFIG_NGINX_SSL \
[OpenWrt-Devel] [PATCH][packages] Enable better customization of nginx features.
Add configuration settings to disable certain features in nginx to be able to reduce it's footprint and allow nginx to be built with support for lua. Signed-off-by: Karl Vogel --- net/nginx/Config.in | 131 + net/nginx/Makefile| 125 +++- net/nginx/patches-lua-nginx/300-ldl.patch | 23 + 3 files changed, 277 insertions(+), 2 deletions(-) create mode 100644 net/nginx/patches-lua-nginx/300-ldl.patch diff --git a/net/nginx/Config.in b/net/nginx/Config.in index e451037..1396d10 100644 --- a/net/nginx/Config.in +++ b/net/nginx/Config.in @@ -36,4 +36,135 @@ config NGINX_DAV help Enable the HTTP and WebDAV methods PUT, DELETE, MKCOL, COPY and MOVE. +config NGINX_MODULE_LUA + bool + prompt "Enable LUA module" + default n + help + Enable support for LUA scripts. + + menu "Disable features" + + config NGINX_DISABLE_PCRE + bool + prompt "Disable PCRE library usage" + default n + + config NGINX_DISABLE_HTTP_CACHE + depends on NGINX_SSL=y + bool + prompt "Disable HTTP cache" + default y + + config NGINX_DISABLE_HTTP_CHARSET_MODULE + bool + prompt "Disable HTTP charset module" + default n + + config NGINX_DISABLE_HTTP_GZIP_MODULE + bool + prompt "Disable HTTP gzip module" + default n + + config NGINX_DISABLE_HTTP_SSI_MODULE + bool + prompt "Disable HTTP ssi module" + default n + + config NGINX_DISABLE_HTTP_USERID_MODULE + bool + prompt "Disable HTTP userid module" + default n + + config NGINX_DISABLE_HTTP_ACCESS_MODULE + bool + prompt "Disable HTTP access module" + default n + + config NGINX_DISABLE_HTTP_AUTH_BASIC_MODULE + bool + prompt "Disable HTTP auth basic" + default n + + config NGINX_DISABLE_HTTP_AUTOINDEX_MODULE + bool + prompt "Disable HTTP autoindex module" + default n + + config NGINX_DISABLE_HTTP_GEO_MODULE + bool + prompt "Disable HTTP geo module" + default n + + config NGINX_DISABLE_HTTP_MAP_MODULE + bool + prompt "Disable HTTP map module" + default n + + config NGINX_DISABLE_HTTP_SPLIT_CLIENTS_MODULE + bool + prompt "Disable HTTP split clients" + default n + + config NGINX_DISABLE_HTTP_REFERER_MODULE + bool + prompt "Disable HTTP referer module" + default n + + config NGINX_DISABLE_HTTP_REWRITE_MODULE + bool + prompt "Disable HTTP rewrite module" + default n + + config NGINX_DISABLE_HTTP_PROXY_MODULE + bool + prompt "Disable HTTP proxy module" + default n + + config NGINX_DISABLE_HTTP_FASTCGI_MODULE + bool + prompt "Disable HTTP fastcgi module" + default n + + config NGINX_DISABLE_HTTP_UWSGI_MODULE + bool + prompt "Disable HTTP uwsgi module" + default n + + config NGINX_DISABLE_HTTP_SCGI_MODULE + bool + prompt "Disable HTTP scgi module" + default n + + config NGINX_DISABLE_HTTP_MEMCACHED_MODULE + bool + prompt "Disable HTTP memcached module" + default n + + config NGINX_DISABLE_HTTP_LIMIT_CONN_MODULE + bool + prompt "Disable HTTP limit conn" + default n + + config NGINX_DISABLE_HTTP_LIMIT_REQ_MODULE + bool + prompt "Disable HTTP limit req" + default n + + config NGINX_DISABLE_HTTP_EMPTY_GIF_MODULE + bool + prompt "Disable HTTP empty gif" + default n + + config NGINX_DISABLE_HTTP_BROWSER_MODULE + bool + prompt "Disable HTTP browser module" + default n + + config NGINX_DISABLE_HTTP_UPSTREAM_IP_HASH_MODULE + bool + prompt "Disable HTTP IP hash module" + default n + + endmenu endmenu diff --git a/net/nginx/Makefile b/net/nginx/Makefile index d56a729..e6fcbc8 100644 --- a/net/nginx/Makefile +++ b/net/nginx/Makefile @@ -22,7 +22,31 @@ PKG_CONFIG_DEPENDS := \ CONFIG_NGINX_STUB_STATUS \ CONFIG_NGINX_FLV \ CONFIG_NGINX_SSL \
Re: [OpenWrt-Devel] [PATCH][packages] Enable better customization of nginx features.
On Nov 16, 2012 11:16 AM, "Karl Vogel" wrote: > > Add configuration settings to disable certain features in nginx to > be able to reduce it's footprint and allow nginx to be built with > support for lua. > > Signed-off-by: Karl Vogel Sorry for the duplicate spam. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH] ubus: Fix imbalance in lua stack push/pop of values.
The lua getglobal and rawgeti both push a value onto the lua stack, but they weren't being removed by the ubus_method_handler function, thus corrupting the lua stack. In case the specified method wasn't a function, the stack was also corrupted as the method name remained on the stack. Signed-off-by: Karl Vogel Cc: Felix Fietkau --- lua/ubus.c |6 +- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/lua/ubus.c b/lua/ubus.c index 6da935d..0f2338c 100644 --- a/lua/ubus.c +++ b/lua/ubus.c @@ -289,6 +289,8 @@ ubus_method_handler(struct ubus_context *ctx, struct ubus_object *obj, lua_getglobal(state, "__ubus_cb"); lua_rawgeti(state, -1, o->r); lua_getfield(state, -1, method); + lua_remove(state, -2); + lua_remove(state, -2); if (lua_isfunction(state, -1)) { lua_pushlightuserdata(state, req); @@ -297,7 +299,9 @@ ubus_method_handler(struct ubus_context *ctx, struct ubus_object *obj, else ubus_lua_parse_blob_array(state, blob_data(msg), blob_len(msg), true); lua_call(state, 2, 0); - } + } else + lua_pop(state, 1); + return 0; } -- 1.7.1 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH] uloop: Add flag to allow callback to be called on error conditions.
In some conditions, an application is interested in errors happening on a file descriptor and might be able to resolve the issue in the callback function. This patch adds a flag to notify the uloop framework that errors should be passed to the callback function, instead of silently removing the fd from the polling set. Signed-off-by: Karl Vogel Cc: Felix Fietkau --- uloop.c |6 -- uloop.h |2 ++ 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/uloop.c b/uloop.c index 0566d80..d293edb 100644 --- a/uloop.c +++ b/uloop.c @@ -174,7 +174,8 @@ static int uloop_fetch_events(int timeout) if (events[n].flags & EV_ERROR) { u->error = true; - uloop_fd_delete(u); + if (!(u->flags & ULOOP_ERROR_CB)) + uloop_fd_delete(u); } if(events[n].filter == EVFILT_READ) @@ -268,7 +269,8 @@ static int uloop_fetch_events(int timeout) if (events[n].events & (EPOLLERR|EPOLLHUP)) { u->error = true; - uloop_fd_delete(u); + if (!(u->flags & ULOOP_ERROR_CB)) + uloop_fd_delete(u); } if(!(events[n].events & (EPOLLRDHUP|EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP))) { diff --git a/uloop.h b/uloop.h index 98dd818..7564514 100644 --- a/uloop.h +++ b/uloop.h @@ -53,6 +53,8 @@ typedef void (*uloop_process_handler)(struct uloop_process *c, int ret); #define ULOOP_EDGE_DEFER (1 << 5) #endif +#define ULOOP_ERROR_CB (1 << 6) + struct uloop_fd { uloop_fd_handler cb; -- 1.7.1 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] [PATCH 1/5] netifd : Disable netlink auto ack
On Tue, Feb 11, 2014 at 10:29:36AM +0100, Hans Dedecker wrote: > Disable netlink auto ack when doing a delete in the get callback > handler to avoid race conditions resulting into stalled message > on the netlink socket. > > Solves issue reported in https://dev.openwrt.org/ticket/14590 > > Signed-off-by: Karl Vogel > Acked-by: Hans Dedecker > --- > system-linux.c |5 +++-- > 1 files changed, 3 insertions(+), 2 deletions(-) > > diff --git a/system-linux.c b/system-linux.c > index db78240..e1b9924 100644 > --- a/system-linux.c > +++ b/system-linux.c > @@ -456,8 +456,9 @@ static int cb_clear_event(struct nl_msg *msg, void *arg) > hdr->nlmsg_type = type; > hdr->nlmsg_flags = NLM_F_REQUEST; > > - if (!nl_send_auto_complete(sock_rtnl, clr->msg)) > - nl_wait_for_ack(sock_rtnl); > + nl_socket_disable_auto_ack(sock_rtnl); > + nl_send_auto_complete(sock_rtnl, clr->msg); > + nl_socket_enable_auto_ack(sock_rtnl); > > return NL_SKIP; > } > -- > 1.7.1 > Just some more background information regarding this.. The current code had 2 issues. First issue was an incorrect return code check, nl_send_auto_complete() returns the number of bytes send or a negative value for an error. Therefor the wait for ack nl_wait_for_ack() was never called in the original code. The second issue is that the incoming netlink message is a multipart message, so if the nl_wait_for_ack() was called, it would get the next part of the multipart message, instead of the actual ACK. Since the original code never called the wait for ack, it didn't run into this issue, but it did receive a stray ACK further down the road, which caused other issues (like the netlink socket not being totally read out, hence why the /proc/net/netlink Rmem column showed outstanding buffered message on the socket). This patch disables the auto ack generation for the delete, since the ack isn't really useful (what would we do if the delete fails anyway?). The only other way to work around this, would be to use another netlink socket to use in the callback, but that requires alot more resources. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] ubus_invoke & interruptions
On Tue, Mar 18, 2014 at 4:06 PM, Felix Fietkau wrote: > > On 2014-03-17 16:49, Helmut Schaa wrote: > > On Mon, Mar 17, 2014 at 4:26 PM, Felix Fietkau wrote: > >> On 2014-03-17 09:18, Helmut Schaa wrote: > >>> just realized that using ubus_invoke can get interrupted by other > >>> uloop-events > >>> happening while executing a ubus method. > >>> > >>> Is this intended? Would it make sense to at least postpone all piled up > >>> events > >>> to be executed "later"? > >> Yes, this is intended. Postponing events in uloop would be a lot more > >> complex than doing the same in the application using it. > > > > Fair enough. Being unaware of this can cause funny effects :D > > > >> uloop would have to handle blocking level-trigger events until they can > >> be handled again, it would have to know which events are needed by > >> libubus for internal processing of the pending request, etc. > > > > I though about adding something like a block_all_events_except_bla > > to libubox and use it in libubus while waiting for the ubus method status > > (if called with a special flag or so). > I thought about this some more, and I think i'll rework libubus to no > longer call uloop_run() for blocking calls. I'll make it use poll() instead. > I think this might also be useful for the lua mapping, because I think what happens in there isn't always 100% correct. ie. if you register a callback from lua and then issue a ubus call from the callback function, it's possible that the ubus call will generate a new callback before the current lua function has finished. I'm not sure, but I think this isn't allowed in the lua vm. It was also the source of a previous bug, which was fixed by a patch I send before: [OpenWrt-Devel] [PATCH] ubus: Fix imbalance in lua stack push/pop of values. https://lists.openwrt.org/pipermail/openwrt-devel/2014-February/023790.html http://nbd.name/gitweb.cgi?p=luci2/ubus.git;a=commit;h=334c38918063b5ece164624e20490021b2bff38a Karl ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] [PATCH] Allow installation of symlinks using RAMFS_COPY_BIN env var.
run_ramfs() uses install_bin to copy executable files to the ram partition. The install_bin function takes multiple parameters, the first being the executable to copy and the rest of the args are symlinks to that binary. There currently isn't any way to pass multiple arguments to install_bin in run_ramfs using the RAMFS_COPY_BIN parameter. This patch enables specifying multiple parameters by separating them with a colon. In effect one can now install a binary and it's symlinks using RAMFS_COPY_BIN. For example: RAMFS_COPY_BIN="/bin/busybox:/bin/sed:/bin/mktemp" Will install 2 symlinks for sed and mktemp to the busybox binary in the ramfs. Without this patch, the sed and mktemp would be complete copies of busybox, taking up precious ram memory. --- package/base-files/files/lib/upgrade/common.sh |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/package/base-files/files/lib/upgrade/common.sh b/package/base-files/files/lib/upgrade/common.sh index e53e844..31857a8 100644 --- a/package/base-files/files/lib/upgrade/common.sh +++ b/package/base-files/files/lib/upgrade/common.sh @@ -60,7 +60,7 @@ run_ramfs() { # [...] install_bin /sbin/fs-state install_bin /sbin/snapshot for file in $RAMFS_COPY_BIN; do - install_bin $file + install_bin ${file//:/ } done install_file /etc/resolv.conf /lib/functions.sh /lib/functions.sh /lib/upgrade/*.sh $RAMFS_COPY_DATA -- 1.7.1 ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] CONFIG_DEBUG and CONFIG_USE_SSTRIP
Currently when CONFIG_DEBUG is enabled, stripping is disabled and the sstrip option becomes unavailable. Looking at the commit history I noticed ticket #6373 which introduced some of these changes. Reading the ticket it seems the intend was to disable stripping when a user selected CONFIG_DEBUG. However the combination of debug + stripping is useful to create a minimal build but still have the option to remote gdb or examine core dumps, ie where the target has stripped binaries, but the unstripped binaries are available on the host/dev system. So I was wondering if it was intentional to remove the sstrip option (as GNU strip is still available) to work around bugs or issues with this combination? or was it an unfortunate side effect of fixing this ticket? Karl ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel