Re: [vdr] epgsearch plugin memory leak
Hi Richard, thanks for the files. I just gave them a try and using valgrind I could also see that there's a leak when a search timer update is run. ==4060== 452,152 (6,968 direct, 445,184 indirect) bytes in 134 blocks are definitely lost in loss record 287 of 287 ==4060==at 0x4024F20: malloc (vg_replace_malloc.c:236) ==4060==by 0x4C31C90: ??? ==4060==by 0x4C39596: ??? ==4060==by 0x4C9FCF2: ??? ==4060==by 0x8145D96: cThread::StartThread(cThread*) (thread.c:257) ==4060==by 0x406D96D: start_thread (pthread_create.c:300) ==4060==by 0x4342A4D: clone (clone.S:130) But unfortunately valgrind does not give me the information where the leak is caused. Maybe the reason for this is that a search timer update runs in a separate thread or because plugins are dynamically loaded shared libraries? Has anybody an idea how to use valgrind in this case? Cheers, Christian Am 20.08.2011 18:13, schrieb Richard F: Hi Christian, To answer your questions... My server is headless - I user Vomp and a pair of media mvp's. I'm using vdr V1.6.0.2 (SD only), I use vdradmin-AM and my epgsearch updates are just on a timer, every 15 mins. Used to be 30 mins but I don't think this makes a significant difference looking at the memory graphs. I've disabled epgsearch conflict checks on timers - I used to have them enabled, but as I look at vdradmin regularly, I can do it manually, and it only fills up the logs. I also use xmltv2vdr on a daily cron to update the epg with full programme details. I attach a tar'd up copy of all my /etc/vdr configs for you, and zipped epg.data. If this doesn't show the problem, perhaps you can tell me how to setup "valgrind" to search for the problem ? Thanks for your help and a great plugin! cheers Richard On 20/08/2011 14:02, Christian Wieninger wrote: Hi Richard, unfortunately I'm not aware of any leaks, if so I would have fixed them ;) Perhaps you can find out in which circumstances the memory usage increases. Candidates are: - search timer updates, should not matter how you trigger them, e.g. via svdrpsend.pl plug epgsearch upds - timer conflict check - simply navigating through the epgsearch menus in OSD Using valgrind would also be a good way to search for the leaks. I've done this before too. But the leak may also depend on your personal usage of epgsearch (e.g. search timer settings, blacklists,...), and therefore did not appear in my valgrind checks. Feel free to send me your VDR configuration (*.conf, epgsearch-config-dir, epg.data) for testing. Cheers, Christian Am 20.08.2011 12:18, schrieb Richard F: Hi, [ Apologies for using this list - the epgsearch bugtracker is broken ] I've been tracking down memory leaks on my server and narrowed down a significant leak in the epgsearch plugin Approx 500 megs / week is lost - recovered by restarting vdr. I'd like to reduce this obviously. I updated from 0.9.24 to 0.9.25 beta 22 from git as a fix for a memory leak was mentioned in the changelog, but the leak is still about the same. If I remove the plugin from the command line, there is no significant leak. Attached plot shows. Anyone have any ideas? Thanks Richard ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] epgsearch plugin memory leak
On 21.08.2011 10:26, Christian Wieninger wrote: > But unfortunately valgrind does not give me the information where the leak > is caused. Maybe the reason for this is that a search timer update runs in > a separate thread or because plugins are dynamically loaded shared libraries? > Has anybody an idea how to use valgrind in this case? http://www.e-tobi.net/blog/2006/04/30/speicherlecks-im-vdr-finden You must keep the plugins loaded, otherwise you loose their symbol information. Here's what I have in the Debian package to support this: http://anonscm.debian.org/gitweb/?p=pkg-vdr-dvb/vdr.git;a=blob;f=debian/patches/82_valgrind.dpatch http://anonscm.debian.org/gitweb/?p=pkg-vdr-dvb/vdr.git;a=blob;f=debian/valgrind.supp http://anonscm.debian.org/gitweb/?p=pkg-vdr-dvb/vdr.git;a=blob;f=debian/vdrleaktest Tobias ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] FreeviewHD success with Nanostick 290e
On Sunday 21 August 2011 02:35:53 you wrote: > I've finally managed to tune my 290e in as well. I have discovered that I > need to remove all of the 2-way splitters that I'd been using to connect > everything up to the wall socket, and then w_scan and scandvb can find the > HD signal. But my version of ffmpeg (0.6.3) is struggling to decode the > stream and so I'm not getting any sound. The pictures are OK, though. So > maybe it's just that this PC is slightly underpowered? (2x2.66GHz P4 Xeons > with HT enabled). I'll try using a beefier machine tomorrow. For me, it was actually very easy to get going: - install drivers (I'm currently on the ones pulled in by the media_build script but I htink the vanilla 3.0.x kernel drivers would have worked) - create a fake channel on the frequency of the HD mux and with QAM256 (I think vdr could find the new mux because I'd been getting loads of "can't tune to channel 0" messages for a while which I put down to the HD mux and not having anything that could tune to it at the time: maybe it would have "just worked" given time) - tune to the fake channel and let vdr add the real channels - delete the fake channel! My current vdr box isn't beefy enough to play back HD recordings (2.7 MHz P4) but it can happily record them and I can play back on my "main" PC using a vdpau-enabled Nvidia card. > And if I'm still getting a HD signal tomorrow, I'll know that my 2-way > splitters really are the problem too :-). Although the loss of the > splitters will be a problem in the long-term, because now I'm limited to > only one DVB-T/T2 receiver. My 290e doesn't want to tune to any of the SD muxes at all but works fine on the HD one! Could be faulty but if it works OK for HD it's fine for me! The signal from my aerial goes through an amplifier to another amplifier with four outputs! I could probably do with a new aerial!! The problem I currently have is the encoded epg from the HD channels. I've got eepg running now but I htink that only processes EPG from the DVB stream rather than processing existing EPG data. This is a problem because I'm downloading EPG from Radio Times and feeding it in using xmltv2vdr. Even so, I don't think eepg was unencoding any new EPG data. The epgsearch plugin manages to find A LOT of recordings when your EPG is garbled!! :-s One step at a time... Cheers, Laz ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [ANNOUNCE] VDR developer version 1.7.20
Am 20.08.2011 12:16, schrieb Klaus Schmidinger: > On 19.08.2011 20:48, Udo Richter wrote: >> Sure, however QueueMessage does not wait and does not return an user key >> press. > > That's by design ;-) > A background thread is not supposed to do this! Osdserver has no other choice. From the main thread, Osdserver could only react once a second, meaning that an average osdserver menu would need 30 seconds to appear. Thus, a completely backgrounded network thread is a must. Osdserver solves this with some complex own locking mechanisms, dirty tricks to force a wakeup of the VDR main thread, and threadsafe shadow copies of non-threadsafe data structures. A global lock would make things a lot easier. >> Osdserver exports all of the message functions, including >> QueueMessage. > > Well, I guess it shouldn't. Would mean, osdserver clients cannot do a simple yes/no query. Not an option. > The kernel developers only recently got rid of this, and they had good > reasons to do so, I guess. I wouldn't want to do that in VDR. You're free to implement a fine-grained locking for all non-threadsafe data structures, like the kernel guys did, but IMHO one big lock is better than none. ;) Anyway, Osdserver is fixed now, and uses a callback from MainThreadHook to do the Skins.Message() call. Cheers, Udo ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] FreeviewHD success with Nanostick 290e
> The signal from my aerial goes through an amplifier to another amplifier with four outputs! > I could probably do with a new aerial!! Interesting, it hadn't occurred to me to use an amplifier. Although the HD MUXs are all broadcasting at low power at the moment until Digital Switch-Over is finished, so it's possible that this problem will sort itself out then. But right now, it looks like my splitters degrade the HD signal to the point where the tuner refuses to lock. Maybe I could buy better splitters, or maybe putting splitters on coaxial cables is not as "plug and play" as I'd supposed and my tangled pile of wires was generating interference. One other point: your amplifier arrangement seems rather sophisticated. Are you *sure* it's sending the same amplified signal to each of its four outputs? This might explain why your 290e can't find the SD MUXs. Cheers, Chris ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
[vdr] ANNOUNCE vdr-ttxtsubs 0.2.3 for VDR 1.7.20
Catching up with some bug reports/feature requests and supporting VDR 1.7.20. The changes: * Made changes in font settings be applied instantly, without the need to change the channel (patch provided by Rolf Ahrenberg) * Converted *.po to UTF-8 (Thx to Ville Skyttä!) * Updated patch for VDR 1.7.20 (Closes #689) * Don't log all page info received from VDR (Closes #331) * Show subtitles with backround rectangle, if outline width is < 2 (patch provided by Mike Lampard) (Closes #247) As always: Any help is welcome! Development site: http://projects.vdr-developer.org/projects/plg-ttxtsubs Downloads: http://projects.vdr-developer.org/projects/plg-ttxtsubs/files Git-Web: http://projects.vdr-developer.org/git/vdr-plugin-ttxtsubs.git/ Anonymous Git-access : git://projects.vdr-developer.org/vdr-plugin-ttxtsubs.git The VDR patch is also available here: git://projects.vdr-developer.org/vdr-patches.git ...which is based on: git://projects.vdr-developer.org/vdr.git This is intended to be a community maintained project! Don't expect me to fix your problems, I'm merely maintaining the project! Please report any bugs, ideas or feature requests to the project site (no registration required for this!). If you want to contribute patches, new features or whatever, post an issue or patch to the projects issue tracker or request to join the project. I would happily add everyone as a project member, who would like to contribute to the project! Tobias ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
[vdr] ANNOUNCE TV-Anytime plugin
A plugin version of my TV-Anytime patch is now available at http://projects.vdr-developer.org/projects/vdrtva The plugin has (most of) the same features as the patch version, but without the need to run a cron job to process the series links. Comments criticisms and suggestions are always welcome... Dave ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [feature request] recording length computation and storage
On 19.08.2011 23:56, Steffen Barszus wrote:
On Fri, 19 Aug 2011 22:18:03 +0200
Udo Richter wrote:
Am 19.08.2011 15:30, schrieb Steffen Barszus:
On 08/19/11 11:46, Steffen Barszus wrote:
i would like to request, that
vdr is storing the length of a recording and make it accessible
to the plug-ins.
Where it gets stored is not my point, that it can be served from in
meory data read by ScanVideoDir is my point.
- read&compute by the core
- dont access harddisk for known recordings
- dont do it multiple times, if more then one part of vdr wants to
know it.
If its only in memory and read by the scanner its fine with me.
for medium to large recording base we speak about minutes not
milliseconds for reading that data.
Maybe some kind of caching mechanism, but please don't calculate the
length every time while doing the scan of the video directory. The
directory scanner is already slow enough, no need to add these minutes
to it.
It's slow since it needs to iterate through all recordings
and check file size of the index file (thats what i understand from
Klaus comment), so divide that by number of frames of the recording and
thats it. the directory scanner is anyway doing the same (iterating
the video directory and pick up necessary information). So it wouldn't
add anything substantially.
Would the attached patch work for you?
Line numbers may be off a little.
Klaus
--- recording.h 2011/08/21 11:34:03 2.24
+++ recording.h 2011/08/21 13:10:39
@@ -89,6 +89,7 @@
mutable char *fileName;
mutable char *name;
mutable int fileSizeMB;
+ mutable int numFrames;
int channel;
int instanceId;
bool isPesRecording;
@@ -123,6 +124,11 @@
int HierarchyLevels(void) const;
void ResetResume(void) const;
double FramesPerSecond(void) const { return framesPerSecond; }
+ int NumFrames(void) const;
+ ///< Returns the number of frames in this recording.
+ ///< If the number of frames is unknown, -1 will be returned.
+ int LengthInSeconds(void) const;
+ ///< Returns the length (in seconds) of this recording, or -1 in case of error.
bool IsNew(void) const { return GetResume() <= 0; }
bool IsEdited(void) const;
bool IsPesRecording(void) const { return isPesRecording; }
--- recording.c 2011/08/21 11:19:55 2.35
+++ recording.c 2011/08/21 13:43:03
@@ -60,6 +60,7 @@
#define DISKCHECKDELTA100 // seconds between checks for free disk space
#define REMOVELATENCY 10 // seconds to wait until next check after removing a file
#define MARKSUPDATEDELTA 10 // seconds between checks for updating editing marks
+#define MININDEXAGE 3600 // seconds before an index file is considered no longer to be written
#define MAX_SUBTITLE_LENGTH 40
@@ -617,6 +618,7 @@
instanceId = InstanceId;
isPesRecording = false;
framesPerSecond = DEFAULTFRAMESPERSECOND;
+ numFrames = -1;
deleted = 0;
// set up the actual name:
const char *Title = Event ? Event->Title() : NULL;
@@ -676,6 +678,7 @@
lifetime = MAXLIFETIME;
isPesRecording = false;
framesPerSecond = DEFAULTFRAMESPERSECOND;
+ numFrames = -1;
deleted = 0;
titleBuffer = NULL;
sortBuffer = NULL;
@@ -1031,6 +1034,25 @@
resume = RESUME_NOT_INITIALIZED;
}
+int cRecording::NumFrames(void) const
+{
+ if (numFrames < 0) {
+ int nf = cIndexFile::GetLength(FileName(), IsPesRecording());
+ if (time(NULL) - LastModifiedTime(FileName()) < MININDEXAGE)
+return nf; // check again later for ongoing recordings
+ numFrames = nf;
+ }
+ return numFrames;
+}
+
+int cRecording::LengthInSeconds(void) const
+{
+ int nf = NumFrames();
+ if (nf >= 0)
+ return int((nf / FramesPerSecond() + 30) / 60) * 60;
+ return -1;
+}
+
// --- cRecordings ---
cRecordings Recordings;
@@ -1102,6 +1124,7 @@
if (endswith(buffer, deleted ? DELEXT : RECEXT)) {
cRecording *r = new cRecording(buffer);
if (r->Name()) {
+ r->NumFrames(); // initializes the numFrames member
Lock();
Add(r);
ChangeState();
@@ -1514,9 +1537,6 @@
// The maximum time to wait before giving up while catching up on an index file:
#define MAXINDEXCATCHUP 8 // seconds
-// The minimum age of an index file for considering it no longer to be written:
-#define MININDEXAGE3600 // seconds
-
struct tIndexPes {
uint32_t offset;
uchar type;
___
vdr mailing list
vdr@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] epgsearch plugin memory leak
Hi, @Tobi: many thanks, worked like a charm! @Richard: I've found a big leak besides some smaller ones and fixed them in the git. So please give it a try and let me know if it workes for you now. Cheers, Christian Am 21.08.2011 10:58, schrieb Tobi: On 21.08.2011 10:26, Christian Wieninger wrote: But unfortunately valgrind does not give me the information where the leak is caused. Maybe the reason for this is that a search timer update runs in a separate thread or because plugins are dynamically loaded shared libraries? Has anybody an idea how to use valgrind in this case? http://www.e-tobi.net/blog/2006/04/30/speicherlecks-im-vdr-finden You must keep the plugins loaded, otherwise you loose their symbol information. Here's what I have in the Debian package to support this: http://anonscm.debian.org/gitweb/?p=pkg-vdr-dvb/vdr.git;a=blob;f=debian/patches/82_valgrind.dpatch http://anonscm.debian.org/gitweb/?p=pkg-vdr-dvb/vdr.git;a=blob;f=debian/valgrind.supp http://anonscm.debian.org/gitweb/?p=pkg-vdr-dvb/vdr.git;a=blob;f=debian/vdrleaktest Tobias ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] epgsearch plugin memory leak
@Christian: I've cloned the Git repo and compiled / running your latest epgsearch with the same config I sent you. Will monitor for a couple of days and let you know, Thanks - R On 21/08/2011 15:50, vdr-requ...@linuxtv.org wrote: Subject: Re: [vdr] epgsearch plugin memory leak From: Christian Wieninger Date: 21/08/2011 15:50 To: VDR Mailing List Hi, @Tobi: many thanks, worked like a charm! @Richard: I've found a big leak besides some smaller ones and fixed them in the git. So please give it a try and let me know if it workes for you now. Cheers, Christian ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] Fix for recording problem in VDR 1.7.20
Hi Klaus,
On 19.08.2011 18:43, Klaus Schmidinger wrote:
There have been some reports about recording problems with VDR 1.7.20
on some HD channels.
This patch should fix this.
Klaus
--- remux.c 2011/08/15 09:50:14 2.58
+++ remux.c 2011/08/19 15:33:26
@@ -974,8 +974,10 @@
payloadUnitOfFrame = (payloadUnitOfFrame + 1) % -framesPerPayloadUnit;
if (payloadUnitOfFrame != 0 && independentFrame)
payloadUnitOfFrame = 0;
- if (payloadUnitOfFrame)
+ if (payloadUnitOfFrame) {
+ newPayload = false;
newFrame = false;
+ }
}
if (framesPerPayloadUnit <= 1)
scanning = false;
Would the log messages look like this without above patch?
Aug 21 16:15:12 vdr vdr: [3138] frame type not in first packet of
payload - buffering
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type
buffer (23312 > 940) - dropped 23124 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: encountered new payload while
buffering - dropping some data!
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type
buffer (3948 > 940) - dropped 3572 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type
buffer (24816 > 940) - dropped 24440 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: encountered new payload while
buffering - dropping some data!
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type
buffer (26696 > 940) - dropped 26508 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: encountered new payload while
buffering - dropping some data!
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type
buffer (20492 > 940) - dropped 20116 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: encountered new payload while
buffering - dropping some data!
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type
buffer (20492 > 940) - dropped 20304 bytes
Regards
André
___
vdr mailing list
vdr@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] Fix for recording problem in VDR 1.7.20
On 21.08.2011 22:20, André Weidemann wrote:
Hi Klaus,
On 19.08.2011 18:43, Klaus Schmidinger wrote:
There have been some reports about recording problems with VDR 1.7.20
on some HD channels.
This patch should fix this.
Klaus
--- remux.c 2011/08/15 09:50:14 2.58
+++ remux.c 2011/08/19 15:33:26
@@ -974,8 +974,10 @@
payloadUnitOfFrame = (payloadUnitOfFrame + 1) % -framesPerPayloadUnit;
if (payloadUnitOfFrame != 0 && independentFrame)
payloadUnitOfFrame = 0;
- if (payloadUnitOfFrame)
+ if (payloadUnitOfFrame) {
+ newPayload = false;
newFrame = false;
+ }
}
if (framesPerPayloadUnit <= 1)
scanning = false;
Would the log messages look like this without above patch?
Aug 21 16:15:12 vdr vdr: [3138] frame type not in first packet of payload -
buffering
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type buffer (23312
> 940) - dropped 23124 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: encountered new payload while buffering
- dropping some data!
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type buffer (3948
> 940) - dropped 3572 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type buffer (24816
> 940) - dropped 24440 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: encountered new payload while buffering
- dropping some data!
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type buffer (26696
> 940) - dropped 26508 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: encountered new payload while buffering
- dropping some data!
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type buffer (20492
> 940) - dropped 20116 bytes
Aug 21 16:15:12 vdr vdr: [3138] ERROR: encountered new payload while buffering
- dropping some data!
Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type buffer (20492
> 940) - dropped 20304 bytes
Those were the reports I got from users.
Klaus
___
vdr mailing list
vdr@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
[vdr] eepg plugin with UK freeview (not freesat!) EPG
Does anyone know whether the eepg plugin should "just work" with the Huffman encoded EPG broadcast on Freeview (DVB-T) HD channels in the UK? (Someone mentioned it on here a week or so back.) I'm pretty sure the EPG data is encoded in exactly the same way as it is on Freesat, i.e. DVB-S, and eepg can decode that EPG. I'm trying to work out whether it should currently be able to do it (and I need to configure something before it'll work), or whether I should try to add support for it. A quick scan of the source code looks like the terms "freesat" and "freeview" are used interchangably in it. Currently, I have completely garbled EPG on HD channels! I can add decoded EPG grabbed via xmltv but the EPG gets overwritten with the garbled version. (What's vdr's policy on EPG data added through SVDRP, etc., being overwritten by data from the EIT?) Cheers, Laz ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] Fix for recording problem in VDR 1.7.20
On Sun, Aug 21, 2011 at 1:20 PM, André Weidemann wrote: > Would the log messages look like this without above patch? > > Aug 21 16:15:12 vdr vdr: [3138] frame type not in first packet of payload - > buffering > Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type buffer > (23312 > 940) - dropped 23124 bytes > Aug 21 16:15:12 vdr vdr: [3138] ERROR: encountered new payload while > buffering - dropping some data! > Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type buffer > (3948 > 940) - dropped 3572 bytes > Aug 21 16:15:12 vdr vdr: [3138] ERROR: too many bytes for frame type buffer > (24816 > 940) - dropped 24440 bytes I had massive amounts of those log messages and all of my recordings were empty. This was fixed by the patch Klaus posted. I expect the same will be true for you as well after applying the patch. Only one way to find out! ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
