Public bug reported: I have apt configured to load a wide variety of sources; my apt is using a local squid-deb-proxy on the same system, and the source that is failing is hosted on an archive mirror on my LAN.
Today I noticed unexpected results from apt-get update: # apt-get update Hit:1 http://wopr/ubuntu bionic InRelease Hit:2 http://wopr/ubuntu bionic-updates InRelease [...] Get:45 http://archive.canonical.com/ubuntu precise Release.gpg [181 B] Hit:46 http://ftp.debian.org/debian stretch Release Ign:26 http://wopr/ubuntu precise Release.gpg Ign:49 http://wopr/ubuntu precise/restricted Sources Ign:50 http://wopr/ubuntu precise/multiverse Sources Ign:48 http://wopr/ubuntu precise/main Sources Hit:49 http://wopr/ubuntu precise/restricted Sources Ign:50 http://wopr/ubuntu precise/multiverse Sources Hit:51 http://wopr/ubuntu precise/universe Sources Ign:51 http://wopr/ubuntu precise/universe Sources Err:50 http://wopr/ubuntu precise/multiverse Sources Could not open file /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 - open (13: Permission denied) [IP: 127.0.0.1 8000] Ign:29 http://ftp.debian.org/debian unstable InRelease Ign:48 http://wopr/ubuntu precise/main Sources Ign:34 http://ftp.debian.org/debian testing InRelease Ign:45 http://archive.canonical.com/ubuntu precise Release.gpg Hit:55 http://archive.canonical.com/ubuntu precise/partner Sources Ign:55 http://archive.canonical.com/ubuntu precise/partner Sources Hit:55 http://archive.canonical.com/ubuntu precise/partner Sources Fetched 670 kB in 7s (101 kB/s) Reading package lists... Done W: GPG error: http://wopr/ubuntu precise-updates InRelease: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 W: GPG error: http://wopr/ubuntu precise-security InRelease: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 W: GPG error: http://wopr/ubuntu precise-proposed InRelease: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 W: GPG error: http://wopr/ubuntu precise Release: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 W: GPG error: http://ftp.debian.org/debian unstable InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7638D0442B90D010 NO_PUBKEY 04EE7237B7D453EC W: GPG error: http://ftp.debian.org/debian testing InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7638D0442B90D010 NO_PUBKEY 04EE7237B7D453EC W: GPG error: http://archive.canonical.com/ubuntu precise Release: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 E: Failed to fetch http://wopr/ubuntu/dists/precise/multiverse/source/Sources Could not open file /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 - open (13: Permission denied) [IP: 127.0.0.1 8000] E: Failed to fetch http://wopr/ubuntu/dists/precise/universe/source/Sources.gz BZ2_bzread: /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_universe_source_Sources.bz2 Read error (-5: DATA_ERROR_MAGIC) E: Some index files failed to download. They have been ignored, or old ones used instead. The unexpected lines are the E: lines. The permission denied line doesn't make sense to me because the permissions *after* the apt-get update run look fine: # namei -l /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 f: /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 drwxr-xr-x root root / drwxr-xr-x root root var drwxr-xr-x root root lib drwxr-xr-x root root apt drwxr-xr-x root root lists drwx------ _apt root partial lrwxrwxrwx root root wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 -> /var/lib/apt/lists/wopr_ubuntu_dists_precise_multiverse_source_Sources drwxr-xr-x root root / drwxr-xr-x root root var drwxr-xr-x root root lib drwxr-xr-x root root apt drwxr-xr-x root root lists -rw-r--r-- root root wopr_ubuntu_dists_precise_multiverse_source_Sources The DATA_ERROR_MAGIC makes enough sense -- the symlink name has .bz2, but the file it points to is a plaintext file: 2f0deae62e2cf7e5257bbd858cb0bf2a94122c4eb82be13e13768d0b9ce84c9e /var/lib/apt/lists/wopr_ubuntu_dists_precise_multiverse_source_Sources This hash is listed in my mirror's Release file: $ grep 2f0deae62e2cf7e5257bbd858cb0bf2a94122c4eb82be13e13768d0b9ce84c9e Release 2f0deae62e2cf7e5257bbd858cb0bf2a94122c4eb82be13e13768d0b9ce84c9e 628753 multiverse/source/Sources My squid-deb-proxy lines matching precise.*Source: 1553540688.354 1 127.0.0.1 TCP_MISS/416 342 GET http://wopr/ubuntu/dists/precise/main/source/Sources.bz2 - HIER_DIRECT/192.168.0.8 text/html 1553540688.406 1 127.0.0.1 TCP_REFRESH_UNMODIFIED_ABORTED/200 8167 GET http://wopr/ubuntu/dists/precise/multiverse/source/Sources.bz2 - HIER_DIRECT/192.168.0.8 application/octet-stream 1553540688.407 0 127.0.0.1 TCP_MISS/416 343 GET http://wopr/ubuntu/dists/precise/universe/source/Sources.bz2 - HIER_DIRECT/192.168.0.8 text/html 1553540688.646 0 127.0.0.1 TCP_MISS/304 315 GET http://wopr/ubuntu/dists/precise/restricted/source/Sources.gz - HIER_DIRECT/192.168.0.8 - 1553540688.648 0 127.0.0.1 TCP_REFRESH_UNMODIFIED_ABORTED/200 8168 GET http://wopr/ubuntu/dists/precise/multiverse/source/Sources.gz - HIER_DIRECT/192.168.0.8 application/octet-stream 1553540688.649 0 127.0.0.1 TCP_MISS/304 317 GET http://wopr/ubuntu/dists/precise/universe/source/Sources.bz2 - HIER_DIRECT/192.168.0.8 - 1553540688.658 0 127.0.0.1 TCP_MISS/416 343 GET http://wopr/ubuntu/dists/precise/main/source/Sources.gz - HIER_DIRECT/192.168.0.8 text/html 1553540688.707 0 127.0.0.1 TCP_MISS/416 343 GET http://wopr/ubuntu/dists/precise/universe/source/Sources.gz - HIER_DIRECT/192.168.0.8 text/html 1553540690.682 312 127.0.0.1 TCP_MISS/416 752 GET http://archive.canonical.com/ubuntu/dists/precise/partner/source/Sources.bz2 - HIER_DIRECT/91.189.92.150 text/html 1553540690.840 156 127.0.0.1 TCP_MISS/304 275 GET http://archive.canonical.com/ubuntu/dists/precise/partner/source/Sources.bz2 - HIER_DIRECT/91.189.92.150 - 1553540690.998 155 127.0.0.1 TCP_MISS/304 275 GET http://archive.canonical.com/ubuntu/dists/precise/partner/source/Sources.gz - HIER_DIRECT/91.189.92.150 - My archive mirror nginx lines matching precise.*Source: 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/main/source/Sources.bz2 HTTP/1.1" 416 222 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/multiverse/source/Sources.bz2 HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/universe/source/Sources.bz2 HTTP/1.1" 416 222 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/restricted/source/Sources.gz HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/multiverse/source/Sources.gz HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/universe/source/Sources.bz2 HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/main/source/Sources.gz HTTP/1.1" 416 222 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/universe/source/Sources.gz HTTP/1.1" 416 222 "-" "Debian APT-HTTP/1.3 (1.6.10)" Thanks ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: apt 1.6.10 ProcVersionSignature: Ubuntu 4.15.0-45.48-generic 4.15.18 Uname: Linux 4.15.0-45-generic x86_64 NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair ApportVersion: 2.20.9-0ubuntu7.6 Architecture: amd64 Date: Mon Mar 25 12:08:10 2019 InstallationDate: Installed on 2012-10-18 (2349 days ago) InstallationMedia: Ubuntu 12.04.1 LTS "Precise Pangolin" - Release amd64 (20120823.1) ProcEnviron: TERM=rxvt-unicode-256color PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: apt UpgradeStatus: Upgraded to bionic on 2018-05-02 (327 days ago) ** Affects: apt (Ubuntu) Importance: Undecided Status: New ** Tags: amd64 apport-bug bionic -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/1821634 Title: BZ2_bzread: [...] Read error (-5: DATA_ERROR_MAGIC) Status in apt package in Ubuntu: New Bug description: I have apt configured to load a wide variety of sources; my apt is using a local squid-deb-proxy on the same system, and the source that is failing is hosted on an archive mirror on my LAN. Today I noticed unexpected results from apt-get update: # apt-get update Hit:1 http://wopr/ubuntu bionic InRelease Hit:2 http://wopr/ubuntu bionic-updates InRelease [...] Get:45 http://archive.canonical.com/ubuntu precise Release.gpg [181 B] Hit:46 http://ftp.debian.org/debian stretch Release Ign:26 http://wopr/ubuntu precise Release.gpg Ign:49 http://wopr/ubuntu precise/restricted Sources Ign:50 http://wopr/ubuntu precise/multiverse Sources Ign:48 http://wopr/ubuntu precise/main Sources Hit:49 http://wopr/ubuntu precise/restricted Sources Ign:50 http://wopr/ubuntu precise/multiverse Sources Hit:51 http://wopr/ubuntu precise/universe Sources Ign:51 http://wopr/ubuntu precise/universe Sources Err:50 http://wopr/ubuntu precise/multiverse Sources Could not open file /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 - open (13: Permission denied) [IP: 127.0.0.1 8000] Ign:29 http://ftp.debian.org/debian unstable InRelease Ign:48 http://wopr/ubuntu precise/main Sources Ign:34 http://ftp.debian.org/debian testing InRelease Ign:45 http://archive.canonical.com/ubuntu precise Release.gpg Hit:55 http://archive.canonical.com/ubuntu precise/partner Sources Ign:55 http://archive.canonical.com/ubuntu precise/partner Sources Hit:55 http://archive.canonical.com/ubuntu precise/partner Sources Fetched 670 kB in 7s (101 kB/s) Reading package lists... Done W: GPG error: http://wopr/ubuntu precise-updates InRelease: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 W: GPG error: http://wopr/ubuntu precise-security InRelease: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 W: GPG error: http://wopr/ubuntu precise-proposed InRelease: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 W: GPG error: http://wopr/ubuntu precise Release: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 W: GPG error: http://ftp.debian.org/debian unstable InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7638D0442B90D010 NO_PUBKEY 04EE7237B7D453EC W: GPG error: http://ftp.debian.org/debian testing InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7638D0442B90D010 NO_PUBKEY 04EE7237B7D453EC W: GPG error: http://archive.canonical.com/ubuntu precise Release: The following signatures were invalid: 630239CC130E1A7FD81A27B140976EAF437D05B5 E: Failed to fetch http://wopr/ubuntu/dists/precise/multiverse/source/Sources Could not open file /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 - open (13: Permission denied) [IP: 127.0.0.1 8000] E: Failed to fetch http://wopr/ubuntu/dists/precise/universe/source/Sources.gz BZ2_bzread: /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_universe_source_Sources.bz2 Read error (-5: DATA_ERROR_MAGIC) E: Some index files failed to download. They have been ignored, or old ones used instead. The unexpected lines are the E: lines. The permission denied line doesn't make sense to me because the permissions *after* the apt-get update run look fine: # namei -l /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 f: /var/lib/apt/lists/partial/wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 drwxr-xr-x root root / drwxr-xr-x root root var drwxr-xr-x root root lib drwxr-xr-x root root apt drwxr-xr-x root root lists drwx------ _apt root partial lrwxrwxrwx root root wopr_ubuntu_dists_precise_multiverse_source_Sources.bz2 -> /var/lib/apt/lists/wopr_ubuntu_dists_precise_multiverse_source_Sources drwxr-xr-x root root / drwxr-xr-x root root var drwxr-xr-x root root lib drwxr-xr-x root root apt drwxr-xr-x root root lists -rw-r--r-- root root wopr_ubuntu_dists_precise_multiverse_source_Sources The DATA_ERROR_MAGIC makes enough sense -- the symlink name has .bz2, but the file it points to is a plaintext file: 2f0deae62e2cf7e5257bbd858cb0bf2a94122c4eb82be13e13768d0b9ce84c9e /var/lib/apt/lists/wopr_ubuntu_dists_precise_multiverse_source_Sources This hash is listed in my mirror's Release file: $ grep 2f0deae62e2cf7e5257bbd858cb0bf2a94122c4eb82be13e13768d0b9ce84c9e Release 2f0deae62e2cf7e5257bbd858cb0bf2a94122c4eb82be13e13768d0b9ce84c9e 628753 multiverse/source/Sources My squid-deb-proxy lines matching precise.*Source: 1553540688.354 1 127.0.0.1 TCP_MISS/416 342 GET http://wopr/ubuntu/dists/precise/main/source/Sources.bz2 - HIER_DIRECT/192.168.0.8 text/html 1553540688.406 1 127.0.0.1 TCP_REFRESH_UNMODIFIED_ABORTED/200 8167 GET http://wopr/ubuntu/dists/precise/multiverse/source/Sources.bz2 - HIER_DIRECT/192.168.0.8 application/octet-stream 1553540688.407 0 127.0.0.1 TCP_MISS/416 343 GET http://wopr/ubuntu/dists/precise/universe/source/Sources.bz2 - HIER_DIRECT/192.168.0.8 text/html 1553540688.646 0 127.0.0.1 TCP_MISS/304 315 GET http://wopr/ubuntu/dists/precise/restricted/source/Sources.gz - HIER_DIRECT/192.168.0.8 - 1553540688.648 0 127.0.0.1 TCP_REFRESH_UNMODIFIED_ABORTED/200 8168 GET http://wopr/ubuntu/dists/precise/multiverse/source/Sources.gz - HIER_DIRECT/192.168.0.8 application/octet-stream 1553540688.649 0 127.0.0.1 TCP_MISS/304 317 GET http://wopr/ubuntu/dists/precise/universe/source/Sources.bz2 - HIER_DIRECT/192.168.0.8 - 1553540688.658 0 127.0.0.1 TCP_MISS/416 343 GET http://wopr/ubuntu/dists/precise/main/source/Sources.gz - HIER_DIRECT/192.168.0.8 text/html 1553540688.707 0 127.0.0.1 TCP_MISS/416 343 GET http://wopr/ubuntu/dists/precise/universe/source/Sources.gz - HIER_DIRECT/192.168.0.8 text/html 1553540690.682 312 127.0.0.1 TCP_MISS/416 752 GET http://archive.canonical.com/ubuntu/dists/precise/partner/source/Sources.bz2 - HIER_DIRECT/91.189.92.150 text/html 1553540690.840 156 127.0.0.1 TCP_MISS/304 275 GET http://archive.canonical.com/ubuntu/dists/precise/partner/source/Sources.bz2 - HIER_DIRECT/91.189.92.150 - 1553540690.998 155 127.0.0.1 TCP_MISS/304 275 GET http://archive.canonical.com/ubuntu/dists/precise/partner/source/Sources.gz - HIER_DIRECT/91.189.92.150 - My archive mirror nginx lines matching precise.*Source: 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/main/source/Sources.bz2 HTTP/1.1" 416 222 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/multiverse/source/Sources.bz2 HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/universe/source/Sources.bz2 HTTP/1.1" 416 222 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/restricted/source/Sources.gz HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/multiverse/source/Sources.gz HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/universe/source/Sources.bz2 HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/main/source/Sources.gz HTTP/1.1" 416 222 "-" "Debian APT-HTTP/1.3 (1.6.10)" 192.168.0.13 - - [25/Mar/2019:12:04:48 -0700] "GET /ubuntu/dists/precise/universe/source/Sources.gz HTTP/1.1" 416 222 "-" "Debian APT-HTTP/1.3 (1.6.10)" Thanks ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: apt 1.6.10 ProcVersionSignature: Ubuntu 4.15.0-45.48-generic 4.15.18 Uname: Linux 4.15.0-45-generic x86_64 NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair ApportVersion: 2.20.9-0ubuntu7.6 Architecture: amd64 Date: Mon Mar 25 12:08:10 2019 InstallationDate: Installed on 2012-10-18 (2349 days ago) InstallationMedia: Ubuntu 12.04.1 LTS "Precise Pangolin" - Release amd64 (20120823.1) ProcEnviron: TERM=rxvt-unicode-256color PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: apt UpgradeStatus: Upgraded to bionic on 2018-05-02 (327 days ago) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1821634/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
