Re: [gentoo-user] HTPC and Gentoo

2012-02-13 Thread Neil Bothwick 
On Mon, 13 Feb 2012 08:50:14 +0100, Raffaele BELARDI wrote:

> I've used the in-tree mythtv for years but lately it wasn't being
> updated [2][3] so I switched to the overlay [4] and been happily running
> 0.25 with that.

> [4] I don't remember which, I have to check. Probably the one mentioned
> in Rich0' blog.

The official overlay is at 
https://github.com/MythTV/packaging/tree/master/Gentoo


-- 
Neil Bothwick

There is no such thing as an atheist in a foxhole.


signature.asc
Description: PGP signature

[gentoo-user] RFC : fast copying of a whole directory tree

2012-02-13 Thread Helmut Jarausch 
Hi,

when copying a whole directory tree with standard tools, e.g.
tar cf - . | ( cd $DEST && tar xf - )
or  cpio -p ...

the source disk is busy seeking. That's noisy and particularly slow.

I've written a small Python program which outputs the file names in
i-node order. If this is fed into tar or cpio nearly no seeks are 
required during copying.

I've tested it by comparing the resulting copied tree to one created by 
tar | tar.

But it's correctness for backing up data is critical.
Therefore I'd like to ask for comments.

Thanks for any comments,
Helmut.
#!/usr/bin/python3
import os, sys, stat


def walktree(top):
'''recursively descend the directory tree rooted at top,
   calling the callback function for each regular file'''
for f in os.listdir(top):
pathname = os.path.join(top, f)
Stat= os.lstat(pathname)
Dev = Stat.st_dev
if  Dev != Root_Dev :
continue
Ino = Stat.st_ino
mode = Stat.st_mode
if stat.S_ISDIR(mode):
# It's a directory, recurse into it
FN_List.append((Ino,pathname))
walktree(pathname)
else :
  FN_List.append((Ino,pathname))


if len(sys.argv) != 2 :
  print('''usage:
  TreeWalk_I_Sorted   # generates a list of files in inode order
  # example with tar :
  TreeWalk_I_Sorted  | tar --no-recursion -c -j -T- -f XXX.tar.bz2
  # example with cpio
  TreeWalk_I_Sorted  | cpio -padmu 
  ''')
  exit(1)

TOP= sys.argv[1]
Stat= os.lstat(TOP)
Root_Dev= Stat.st_dev
FN_List=[(Stat.st_ino,TOP)]

# import resource
# print("at Start in kB ",resource.getrusage(0).ru_maxrss)
# uses about 500 bytes per file

walktree(TOP)
FN_List.sort()

# print("*** starting ...",file=sys.stderr)
for I,F in FN_List :
  print(F)  #  print(I," -> " ,F)

# print("after loading",len(FN_List)," items : ",resource.getrusage(0).ru_maxrss)

Re: [gentoo-user] RFC : fast copying of a whole directory tree

2012-02-13 Thread Michael Orlitzky 
On 02/13/12 05:49, Helmut Jarausch wrote:
> 
> I've written a small Python program which outputs the file names in
> i-node order. If this is fed into tar or cpio nearly no seeks are 
> required during copying.

What makes you think the inodes are sequential on-disk?


> But it's correctness for backing up data is critical.
> Therefore I'd like to ask for comments.

You're nuts =)

Seriously though, use cp, tar, or rsync. They've seen years of use by
millions of people. All of the remaining bugs are sufficiently insidious
that you'll never hit them. The same probably isn't true for your script!

[gentoo-user] Re: RFC : fast copying of a whole directory tree

2012-02-13 Thread Grant Edwards 
On 2012-02-13, Michael Orlitzky  wrote:
> On 02/13/12 05:49, Helmut Jarausch wrote:
>> 
>> I've written a small Python program which outputs the file names in
>> i-node order. If this is fed into tar or cpio nearly no seeks are 
>> required during copying.
>
> What makes you think the inodes are sequential on-disk?

Even if the i-nodes are sequential on-disk, there's no reason to think
that the data blocks associated with the inodes are in any particular
order with respect to the i-nodes themselves.

>> But it's correctness for backing up data is critical.
>> Therefore I'd like to ask for comments.
>
> You're nuts =)
>
> Seriously though, use cp, tar, or rsync. They've seen years of use by
> millions of people. All of the remaining bugs are sufficiently
> insidious that you'll never hit them. The same probably isn't true
> for your script!

-- 
Grant Edwards   grant.b.edwardsYow! All this time I've
  at   been VIEWING a RUSSIAN
  gmail.comMIDGET SODOMIZE a HOUSECAT!

Re: [gentoo-user] KMail

2012-02-13 Thread Jens Reinemuth 
Am Donnerstag, 9. Februar 2012, 18:33:12 schrieb Daniel D Jones:
> I was one of those bitten by the upgrade to KMail back in October.  After
> spending nearly two days without access to email while trying to get the
> upgrade and migration to work, I gave up and pinned KMail to version
> 4.4.11.1.
> 
> With the recent upgrade to KDE, I'm curious if anyone knows whether the bugs
> have been worked out and the KMail upgrade will go smoother now?

Just stay with 4.4.x! Akonadi/Nepomuk/... is imho not ready for daily use 
right now! I got 4.8 running in Kubuntu and its a real pain, even with the 8GB 
Ram in the company-laptop!

In private i will never switch! If someday you can't use the older Versions 
with actual KDE i will use Thunderbird...

Jens

Re: [gentoo-user] Re: RFC : fast copying of a whole directory tree

2012-02-13 Thread Joerg Schilling 
Grant Edwards  wrote:

> On 2012-02-13, Michael Orlitzky  wrote:
> > On 02/13/12 05:49, Helmut Jarausch wrote:
> >> 
> >> I've written a small Python program which outputs the file names in
> >> i-node order. If this is fed into tar or cpio nearly no seeks are 
> >> required during copying.
> >
> > What makes you think the inodes are sequential on-disk?
>
> Even if the i-nodes are sequential on-disk, there's no reason to think
> that the data blocks associated with the inodes are in any particular
> order with respect to the i-nodes themselves.

Correct, there is however a really fast method using "star -copy".

This works because there are two decoupled processes, shared memory between 
them and the fact that star reads names from directories in one big chunk.

Jörg

-- 
 EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
   j...@cs.tu-berlin.de(uni)  
   joerg.schill...@fokus.fraunhofer.de (work) Blog: 
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Re: RFC : fast copying of a whole directory tree

2012-02-13 Thread Pandu Poluan 
On Feb 13, 2012 11:15 PM, "Joerg Schilling" <
joerg.schill...@fokus.fraunhofer.de> wrote:
>
> Grant Edwards  wrote:
>
> > On 2012-02-13, Michael Orlitzky  wrote:
> > > On 02/13/12 05:49, Helmut Jarausch wrote:
> > >>
> > >> I've written a small Python program which outputs the file names in
> > >> i-node order. If this is fed into tar or cpio nearly no seeks are
> > >> required during copying.
> > >
> > > What makes you think the inodes are sequential on-disk?
> >
> > Even if the i-nodes are sequential on-disk, there's no reason to think
> > that the data blocks associated with the inodes are in any particular
> > order with respect to the i-nodes themselves.
>
> Correct, there is however a really fast method using "star -copy".
>
> This works because there are two decoupled processes, shared memory
between
> them and the fact that star reads names from directories in one big chunk.
>

Honestly, that's news to me. Which package has star?

Rgds,

[gentoo-user] Re: RFC : fast copying of a whole directory tree

2012-02-13 Thread Nikos Chantziaras 

On 13/02/12 18:29, Pandu Poluan wrote:


On Feb 13, 2012 11:15 PM, "Joerg Schilling"
mailto:joerg.schill...@fokus.fraunhofer.de>> wrote:
 > Correct, there is however a really fast method using "star -copy".
 >
 > This works because there are two decoupled processes, shared memory
between
 > them and the fact that star reads names from directories in one big
chunk.
 >

Honestly, that's news to me. Which package has star?


eix -e star

:-/

Re: [gentoo-user] Re: RFC : fast copying of a whole directory tree

2012-02-13 Thread Pandu Poluan 
On Feb 13, 2012 11:41 PM, "Nikos Chantziaras"  wrote:
>
> On 13/02/12 18:29, Pandu Poluan wrote:
>>
>>
>> On Feb 13, 2012 11:15 PM, "Joerg Schilling"
>> > > wrote:
>>  > Correct, there is however a really fast method using "star -copy".
>>  >
>>  > This works because there are two decoupled processes, shared memory
>> between
>>  > them and the fact that star reads names from directories in one big
>> chunk.
>>  >
>>
>> Honestly, that's news to me. Which package has star?
>
>
> eix -e star
>
> :-/
>

Hehhe... sorry, I'm on the road and don't have Gentoo on my smartphone :-P

Rgds,

[gentoo-user] Jack, the cat and its tail...

2012-02-13 Thread meino . cramer 
Hi,

I am currently trying to "jackify" my system with jackd and
pulseaudio.

I am also running some kaffeine, therefore the basic kde stuff
is installed (as less as possible). I am running openbox and slim...

The problem is: Kaffeine produces no sound.

Since Kaffeine uses xine I set ~/.kde4/share/apps/kaffeine/xine-config
to use jackd (audio.driver:jack).

This results in nothing audible and the screen freezes after a short
time...

Therefore I tried xine itsself...and patches ~/.xine/config according
to the above.

Same result.

But this time xine prints at the commandline:
This is xine (X11 gui) - a free video player v0.99.6.
(c) 2000-2007 The xine Team.
This system has no accessible HPET device (Device or resource busy)

I used fuser to get the name of the guiltyand gues who it is?

/home/user>fuser /dev/hpet 
 USERPID ACCESS COMMAND
/dev/hpet:   mccramer   4221 f...m (mccramer)jackd

So  I cannot use xine/kaffeine with jackd cause jackd is running?

The cat and its tail...

How can I circumvent this relly nice deadlock?

Best regards,
mcc

Re: [gentoo-user] Re: RFC : fast copying of a whole directory tree

2012-02-13 Thread Joerg Schilling 
Nikos Chantziaras  wrote:

> >  > This works because there are two decoupled processes, shared memory
> > between
> >  > them and the fact that star reads names from directories in one big
> > chunk.
> >  >
> >
> > Honestly, that's news to me. Which package has star?
>
> eix -e star

To help star to buffer, give star a large fifo size that is up to haslf of the 
RAM in your machine, e.g. fs=1000m

To make sure that star gives fast file creation (unpacking of archives) on 
filesystems that do not support fast verified transactions, you need to make 
star as "insecure" as other software to get comparable results, so add:

-no-fsync

Jörg

-- 
 EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
   j...@cs.tu-berlin.de(uni)  
   joerg.schill...@fokus.fraunhofer.de (work) Blog: 
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

[gentoo-user] Raid1 simple workstation

2012-02-13 Thread James 
Hello,

Well it's been almost a year since I tried to set up 
a simple raid1 workstation. (I.E. 2 identical drives,
boo/root/swap only with Raid 1). So I'm starting over.

I found this link:
http://en.gentoo-wiki.com/wiki/RAID/Software

Is this the best document to follow?
(note, I'm going to try to avoid using LVM:

fdisk -l

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xab83344a

   Device Boot  Start End  Blocks   Id  System
/dev/sda1   *2048  526335  262144   fd  Linux raid autodetect
/dev/sda2  52633610573823 5023744   fd  Linux raid autodetect
/dev/sda310573824  3907029167  1948227672   fd  Linux raid autodetect

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xab83344a

   Device Boot  Start End  Blocks   Id  System
/dev/sdb1   *2048  526335  262144   fd  Linux raid autodetect
/dev/sdb2  52633610573823 5023744   fd  Linux raid autodetect
/dev/sdb310573824  3907029167  1948227672   fd  Linux raid autodetect

Disk /dev/md127: 268 MB, 268369920 bytes
2 heads, 4 sectors/track, 65520 cylinders, total 524160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x

Disk /dev/md127 doesn't contain a valid partition table

Re: [gentoo-user] Re: RFC : fast copying of a whole directory tree

2012-02-13 Thread Dale 
Joerg Schilling wrote:
> Nikos Chantziaras  wrote:
> 
>>>  > This works because there are two decoupled processes, shared memory
>>> between
>>>  > them and the fact that star reads names from directories in one big
>>> chunk.
>>>  >
>>>
>>> Honestly, that's news to me. Which package has star?
>>
>> eix -e star
> 
> To help star to buffer, give star a large fifo size that is up to haslf of 
> the 
> RAM in your machine, e.g. fs=1000m
> 
> To make sure that star gives fast file creation (unpacking of archives) on 
> filesystems that do not support fast verified transactions, you need to make 
> star as "insecure" as other software to get comparable results, so add:
> 
>   -no-fsync
> 
> Jörg
> 


The problem with star is that when I need to copy a large number of
files, it isn't on the DVD I boot from.  That's why most people use cp
since it is on every bootable media I have ever booted.  That includes
the Gentoo bootable media.

Since star is so good, why not get them to include it on the bootable
media?  Is it to large a package or what?

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or
how you interpreted my words!

Miss the compile output?  Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

Re: [gentoo-user] Re: RFC : fast copying of a whole directory tree

2012-02-13 Thread Neil Bothwick 
On Tue, 14 Feb 2012 00:42:56 +0700, Pandu Poluan wrote:

> Hehhe... sorry, I'm on the road and don't have Gentoo on my
> smartphone :-P

Not even via SSH? :P


-- 
Neil Bothwick

If at first you don't succeed you'll get lots of advice.


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: RFC : fast copying of a whole directory tree

2012-02-13 Thread Pandu Poluan 
On Feb 14, 2012 6:00 AM, "Neil Bothwick"  wrote:
>
> On Tue, 14 Feb 2012 00:42:56 +0700, Pandu Poluan wrote:
>
> > Hehhe... sorry, I'm on the road and don't have Gentoo on my
> > smartphone :-P
>
> Not even via SSH? :P
>

It's a new phone and I forgot the port-knocking sequence to open the ssh
port >.<

Rgds,

[gentoo-user] eclean-dist error

2012-02-13 Thread »Q« 
This isn't much of a problem for me, but eclean-dist is behaving
strangely for me.  I don't know if it's a bug or if something screwy
I'm not seeing with my exclusion file.

$ eclean-dist --pretend
 * Building file list for distfiles cleaning...
 * Your distfiles directory was already clean.

   The following Deprecated files were saved from cleaning due to exclusion 
file entries
Traceback (most recent call last):
  File "/usr/bin/eclean-dist", line 44, in 
main()
  File "/usr/lib64/python2.7/site-packages/gentoolkit/eclean/cli.py", line 494, 
in main
output=output)
  File "/usr/lib64/python2.7/site-packages/gentoolkit/eclean/cli.py", line 427, 
in doAction
clean_size = cleaner.pretend_clean(saved)
UnboundLocalError: local variable 'cleaner' referenced before assignment

-

The two deprecated files saved from cleaning are openrc-0.9.8.1.tar.bz2
and openrc-0.9.8.2.tar.bz2, which I see (without errors) when I use the
eclean-dist --destructive :

$ eclean-dist --pretend --destructive
 * Building file list for distfiles cleaning...
 * Here are the distfiles that would be deleted:
 [   48.8 K ] simplejson-2.2.1.tar.gz
 [   29.2 M ] wxPython-src-2.8.11.0.tar.bz2
 ===
 [   29.2 M ] Total space from 2 files would be freed in the distfiles directory

   The following Deprecated files were saved from cleaning due to exclusion 
file entries
 [  162.9 K ] openrc-0.9.8.1.tar.bz2
 [  162.9 K ] openrc-0.9.8.2.tar.bz2
 ===
 [  325.9 K ] Total space from 2 package files
   Re-run the last command with the -D option to clean them as well

-

And here's my exclusion file:

$ cat /etc/eclean/distfiles.exclude 
# /etc/eclean/distfiles.exclude
# In this file you can list some categories or cat/pkg-name for which you want
# to protect distfiles from "ecleaning". You can also name some specific files.
# See `man eclean` for syntax details.

metadata.dtd

## categories to keep
sys-apps
app-arch
app-backup
app-portage

## packages to keep

dev-libs/openssl
sys-devel/binutils