On Monday 18 December 2006 16:40, Attila Fülöp wrote:
> Kern Sibbald wrote:
> > On Monday 18 December 2006 15:06, Mair Wolfgang-awm013 wrote:
> >> Ok,
> >>
> >> So here is an update from what I found out so far while playing around
> >> with my Solaris system here.
> >>
> >> A find for doors brings me the following list. As you can see, most of
> >> the doors are located in the /proc.
> >>
> > ...
> >> /proc/387/fd/3
> >> 54 0 D--------- 1 root root 0 Dec 18 12:52
> >> /proc/397/fd/6
> >>
> >> Next I deleted them all:
> >> # find / -type D -exec rm -f {} \;
> >> (Door files in the /proc dir are not able to be deleted)
> >>
> >> After that, the find brings me up only the files in the /proc which is
> >> what I wanted.
> >>
> >> Then reboot.
> >>
> >> After the system came up again I did the same find as above:
> >> # find / -type D -ls
> >> # find / -type D -ls
> >> 60 0 Dr--r--r-- 1 root root 0 Dec 18 13:12
> > ...
> >> 54 0 D--------- 1 root root 0 Dec 18 13:12
> >> /proc/385/fd/6
> >>
> >>
> >> So, as you can see the door files got created automatically. Can this be
> >> taken now as an evidence in general, that ALL needed doors will be
> >> created automatically on demand by the system?
> >> If so, this should be added to the bacula code, that doors have to be
> >> threatened as 'special' files and should not be backed up, as mentioned
> >> by Kern below.
> >
> > OK, but what defines a door. Is it identified by a special bit in the stat
> > packet? If so what bit.
>
> Yes, S_IFDOOR
That is what I had imagined.
>
> > Can you find documentation on it?
>
> stat.h(3HEAD)
> eg http://docs.sun.com/app/docs/doc/816-5173/6mbb8ae23?a=view#indexterm-360
>
> > If it is a
> > special bit, Bacula can avoid backing them up, which would then avoid the
> > problems during the restore. For door files that are already backed up by
> > old software, we could also if they are identifiable, not retore them.
>
> Should be no problem since the stat structure is saved on tape.
> Should I look into it?
Yes, since I am not able to test it easily.
Regards,
Kern
>
> >> Wolfgang
> >>
> >>
> >> -----Original Message-----
> >> From: [EMAIL PROTECTED]
> >> [mailto:[EMAIL PROTECTED] On Behalf Of Richard
> >> Mortimer
> >> Sent: Friday, December 15, 2006 18:59
> >> To: 'Richard Mortimer'; 'Kern Sibbald';
> >> [email protected]
> >> Subject: Re: [Bacula-users] restore on solaris unexpected
> >> freeinode/problems with doors
> >>
> >> Replying to myself after just having looked at the OpenSolaris
> >> sourcecode for devfsadm.
> >>
> >> Anyway devfsadm creates a door file as follows:
> >>
> >> 1 - unlink any existing file
> >> 2 - create an empty regular file with the permissions required for
> >> the door. Then close the file.
> >> 3 - use door_create() to get a door fd and then fattach it to
> >> the filesystem.
> >>
> >> Thus I think that the correct way to restore a door file is to either
> >> (a) ignore it; or (b) create an empty regular file.
> >>
> >> I'm tempted to think that (b) is correct because some other uses of
> >> doors ignore errors creating the file (some even check for existance
> >> beforehand) and just do an fattach afterwards.
> >>
> >> The implementation is left as an exercise for any interested readers who
> >> have Solaris systems!
> >>
> >> Richard
> >>
> >>> -----Original Message-----
> >>> From: [EMAIL PROTECTED]
> >>> [mailto:[EMAIL PROTECTED] On Behalf Of
> >>> Richard Mortimer
> >>> Sent: 15 December 2006 17:36
> >>> To: 'Kern Sibbald'; [email protected]
> >>> Subject: Re: [Bacula-users] restore on solaris unexpected free
> >>> inode/problems with doors
> >>>
> >>> Kern,
> >>>
> >>> Doors are special files that are used for inter-process calls.
> >>> Basically the calling process goes through a door into another (server
> >>> processes) context, does the processing and returns. It is designed as
> >>> a quick interprocess mechanism that doesn't involve a full context
> >>> switch to a different stack context. They work in a similar way to
> >>> fifo where a process associates itself with a filesystem node and
> >>> registers a number of handler threads that do work on their behalf.
> >>>
> >>> I guess that the fact that you haven't heard of doors means that
> >>> bacula hasn't either. I don't remember exactly how you make a door
> >>> filesystem entry and I don't have a Solaris system on hand to take a
> >>> look further.
> >>>
> >>> The man page for door_create(3DOOR) lives at
> >>> http://docs.sun.com/app/docs/doc/816-5171/6mbb6dcne?a=view
> >>> I don't remember whether these need "restoring" although I suspect
> >>> that they do because some of their functionality may be needed early
> >>> on in boot.
> >>>
> >>> Regards
> >>>
> >>> Richard
> >>>
> >>>> -----Original Message-----
> >>>> From: [EMAIL PROTECTED]
> >>>> [mailto:[EMAIL PROTECTED] On Behalf Of
> >>>> Kern Sibbald
> >>>> Sent: 15 December 2006 17:10
> >>>> To: [email protected]
> >>>> Subject: Re: [Bacula-users] restore on solaris unexpected free inode
> >>>> /problems with doors
> >>>>
> >>>> Well, about the only other thing I can think of is that these are
> >>>> some sort of
> >>>> *special* Solaris file that Bacula doesn't know how to properly
> >>>> restore or should not be restoring (sockets?) or there are some sort
> >>>> of access control lists that are interferring with their creation,
> >>>> or they are files in some special system directory that probably
> >>>> should not be restored (e.g. /sys or /proc if it were a Linux
> >>>> system).
> >>>>
> >>>> Taking a careful look at the same files on a running system will
> >>>> probably give you a clue. If they are sockets, you can safely
> >>>> ignore the error messages.
> >>>> Version 1.39.x will not try to restore sockets if I am not
> >>> mistaken.
> >>>> A good test of whether or not the restore worked is trying to boot
> >>>> up the system and run some programs on it -- e.g. a Window system,
> >>>> and build Bacula.
> >>>> Just the same, I would not be comfortable until I understood the
> >>>> nature of those errors ...
> >>>>
> >>>> On Friday 15 December 2006 17:50, Mair Wolfgang-awm013 wrote:
> >>>>> Kern, Richard,
> >>>>>
> >>>>> While the System is still booted from network I did a few
> >>>> sync's prio to
> >>>>> umount the partition. But gave me the same error at boot.
> >>>>>
> >>>>> When I do a fsck after umount but prio to a reboot, I see the same
> >>>>> messages as mentioned below.
> >>>>>
> >>>>> If I cd into the directory before the umount and before the
> >>>> fsck. I get
> >>>>> the following:
> >>>>>
> >>>>> # ls -li sysevent_door
> >>>>> 138043 ?rw------- 1 root root 0 Dec 14 08:57
> >>>>> sysevent_door
> >>>>> #
> >>>>>
> >>>>> As you can see this file can not be identified as door.
> >>>> Mind the '?' it
> >>>>> should be a 'D'.
> >>>>>
> >>>>> The fsck first finds the inode as 'PARTIALLY ALLOCATED
> >>>> INODE' and in the
> >>>>> Phase 2 it deletes these files. Means, if I mount the
> >>>> partition again
> >>>>> after the fsck is finished. These files are gone.
> >>>>>
> >>>>> Also, the boot and fsck messages below seem to be from
> >>> two different
> >>>>> sessions. The inodes displayed at the boot time will come
> >>> up in the
> >>>>> fsck. But fsck finds even more of them. Sorry for that.
> >>>>>
> >>>>> Wolfgang
> >>>>>
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: Kern Sibbald [mailto:[EMAIL PROTECTED]
> >>>>> Sent: Friday, December 15, 2006 16:39
> >>>>> To: [email protected]
> >>>>> Cc: Mair Wolfgang-awm013
> >>>>> Subject: Re: [Bacula-users] restore on solaris unexpected
> >>>> free inode /
> >>>>> problems with doors
> >>>>>
> >>>>> Hello,
> >>>>>
> >>>>> This is an interesting way of going about a bare metal
> >>>> recovery. I have
> >>>>> several suggestions that may help you.
> >>>>>
> >>>>> 1. After everything is restored and you kernel is reinstalled,
> >>>>> do:
> >>>>>
> >>>>> sync; sync; sync
> >>>>>
> >>>>> which will cause the kernel disk cache to be flushed.
> >>>>>
> >>>>> 2. Before rebooting, umount all disks or if they cannot be
> >>>> unmounted,
> >>>>> remount them read-only.
> >>>>>
> >>>>> Hopefully those two items will ensure that all changed
> >>>> information is
> >>>>> written
> >>>>> to the disk before rebooting. Concerning the _door files,
> >>>> I have no
> >>>>> idea
> >>>>> what they are. Hopefully a Solaris expert can answer the
> >>> question.
> >>>>> Regards,
> >>>>>
> >>>>> Kern
> >>>>>
> >>>>> On Friday 15 December 2006 15:57, Mair Wolfgang-awm013 wrote:
> >>>>>> Hello,
> >>>>>>
> >>>>>> I've encountered some problems with my bare metal
> >>>> recovery on solaris
> >>>>>> 9 with bacula 1.38.11.
> >>>>>>
> >>>>>> Here is a brief description of the recovery procedure I
> >>> did (this
> >>>>>> might also be a good description of how to do a bare
> >>>> metal recovery,
> >>>>>> if this issue gets solved):
> >>>>>>
> >>>>>> Boot the client from network in singleuser mode.
> >>>>>> Mount a shared directory containing bacula to the client.
> >>>>>> # mount 192.168.230.10:/export/local /usr/local
> >>>>>>
> >>>>>> Partition the target hard drive.
> >>>>>> # format
> >>>>>>
> >>>>>> Created a new file system on the target partition.
> >>>>>> # newfs /dev/dsk/c0t0d0s0
> >>>>>>
> >>>>>> Mount the new target partition to /a # mount /dev/dsk/c0t0d0s0
> >>>>>> /a
> >>>>>>
> >>>>>> Create the necessary entries for the director and the
> >>>> storage server
> >>>>>> into the hosts file # echo "192.168.230.35 volvo" >>
> >>>> /etc/hosts # echo
> >>>>>> "192.168.230.30 prinz" >> /etc/hosts # echo "192.168.230.10
> >>>>>> automatix3" >> /etc/hosts
> >>>>>>
> >>>>>> Set the LD_LIBRARY_PATH var to where the bacula libs are:
> >>>>>> # LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib
> >>>>>> # export LD_LIBRARY_PATH
> >>>>>>
> >>>>>> Start the bacula file daemon
> >>>>>> # /usr/local/etc/bacula start
> >>>>>> Starting the Bacula File daemon
> >>>>>> #
> >>>>>>
> >>>>>> Now go to the bconsole and start the restore procedure
> >>>> and select all
> >>>>>> files to restore:
> >>>>>> *restore
> >>>>>>
> >>>>>> First you select one or more JobIds that contain files to
> >>>> be restored.
> >>>>>> You will be presented several methods of specifying the
> >>>> JobIds. Then
> >>>>>> you will be allowed to select which files from those
> >>>> JobIds are to be
> >>>>>> restored.
> >>>>>>
> >>>>>> To select the JobIds, you have the following choices:
> >>>>>> 1: List last 20 Jobs run
> >>>>>> 2: List Jobs where a given File is saved
> >>>>>> 3: Enter list of comma separated JobIds to select
> >>>>>> 4: Enter SQL list command
> >>>>>> 5: Select the most recent backup for a client
> >>>>>> 6: Select backup for a client before a specified time
> >>>>>> 7: Enter a list of files to restore
> >>>>>> 8: Enter a list of files to restore before a specified time
> >>>>>> 9: Find the JobIds of the most recent backup for a client
> >>>>>> 10: Find the JobIds for a backup for a client before
> >>>> a specified
> >>>>>> time
> >>>>>> 11: Enter a list of directories to restore for found JobIds
> >>>>>> 12: Cancel
> >>>>>> Select item: (1-12): 5
> >>>>>> Defined Clients:
> >>>>>> 1: passat-fd
> >>>>>> ............
> >>>>>> 47: mondeo-fd
> >>>>>> Select the Client (1-47): 47
> >>>>>> Automatically selected FileSet: FullSolaris
> >>>>>>
> >>> +-------+-------+----------+---------------+---------------------+----
> >>>> +-------+-------+----------+---------------+---------------------+--
> >>> +-------+-------+----------+---------------+---------------------+----
> >>>> +-------+-------+----------+---------------+---------------------+--
> >>>> +-------+-------+----------+---------------+---------------------+--
> >>>>>> ------+-----------+
> >>>>>> | JobId | Level | JobFiles | JobBytes | StartTime
> >>> |
> >>>>>> VolumeName
> >>>>>> | StartFile |
> >>>>>>
> >>> +-------+-------+----------+---------------+---------------------+----
> >>>> +-------+-------+----------+---------------+---------------------+--
> >>> +-------+-------+----------+---------------+---------------------+----
> >>>> +-------+-------+----------+---------------+---------------------+--
> >>>> +-------+-------+----------+---------------+---------------------+--
> >>>>>> ------+-----------+
> >>>>>> | 547 | F | 86,228 | 1,965,081,652 | 2006-12-15
> >>>> 13:22:45 |
> >>>>>> full-15-12-2006.90 | 0 |
> >>>>>>
> >>> +-------+-------+----------+---------------+---------------------+----
> >>>> +-------+-------+----------+---------------+---------------------+--
> >>> +-------+-------+----------+---------------+---------------------+----
> >>>> +-------+-------+----------+---------------+---------------------+--
> >>>> +-------+-------+----------+---------------+---------------------+--
> >>>>>> ------+-----------+
> >>>>>> You have selected the following JobId: 547
> >>>>>>
> >>>>>> Building directory tree for JobId 547 ...
> >>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++
> >>>>>> 1 Job, 83,160 files inserted into the tree.
> >>>>>>
> >>>>>> You are now entering file selection mode where you add
> >>> (mark) and
> >>>>>> remove
> >>>>>> (unmark) files to be restored. No files are initially
> >>>> added, unless
> >>>>>> you used the "all" keyword on the command line.
> >>>>>> Enter "done" to leave this mode.
> >>>>>>
> >>>>>> cwd is: /
> >>>>>> $ mark *
> >>>>>> 86,227 files marked.
> >>>>>> $ done
> >>>>>> Bootstrap records written to
> >>>>>> /var/bacula/working/volvo-dir.8.restore.bsr
> >>>>>>
> >>>>>> The job will require the following Volumes:
> >>>>>>
> >>>>>> full-15-12-2006.90
> >>>>>>
> >>>>>>
> >>>>>> 86227 files selected to be restored.
> >>>>>>
> >>>>>> Run Restore job
> >>>>>> JobName: RestoreFiles
> >>>>>> Bootstrap: /var/bacula/working/volvo-dir.8.restore.bsr
> >>>>>> Where: /
> >>>>>> Replace: always
> >>>>>> FileSet: FullSolaris
> >>>>>> Client: mondeo-fd
> >>>>>> Storage: File
> >>>>>> When: 2006-12-15 15:26:35
> >>>>>> Catalog: MyCatalog
> >>>>>> Priority: 10
> >>>>>> OK to run? (yes/mod/no): m
> >>>>>> Parameters to modify:
> >>>>>> 1: Level
> >>>>>> 2: Storage
> >>>>>> 3: Job
> >>>>>> 4: FileSet
> >>>>>> 5: Client
> >>>>>> 6: When
> >>>>>> 7: Priority
> >>>>>> 8: Bootstrap
> >>>>>> 9: Where
> >>>>>> 10: Replace
> >>>>>> 11: JobId
> >>>>>> Select parameter to modify (1-11): 9 Please enter path prefix
> >>>>>> for restore (/ for none): /a Run
> >>>> Restore job
> >>>>>> JobName: RestoreFiles
> >>>>>> Bootstrap: /var/bacula/working/volvo-dir.8.restore.bsr
> >>>>>> Where: /a
> >>>>>> Replace: always
> >>>>>> FileSet: FullSolaris
> >>>>>> Client: mondeo-fd
> >>>>>> Storage: File
> >>>>>> When: 2006-12-15 15:26:35
> >>>>>> Catalog: MyCatalog
> >>>>>> Priority: 10
> >>>>>> OK to run? (yes/mod/no): y
> >>>>>> Job started. JobId=554
> >>>>>> *
> >>>>>> *
> >>>>>>
> >>>>>> Then go back to the client and:
> >>>>>> # installboot /a/usr/platform/`uname -i`/lib/fs/ufs/bootblk
> >>>>>> /dev/rdsk/c0t0d0s0
> >>>>>>
> >>>>>>
> >>>>>> Until here everything looks fine. But when I try to
> >>> boot from the
> >>>>>> newly created disk I see the following inode messages:
> >>>>>>
> >>>>>>
> >>>>>> Rebooting with command: boot
> >>>>
> >>>>>> Boot device: disk:a File and args:
> >>>>>> SunOS Release 5.9 Version Generic_118558-11 64-bit
> >>>> Copyright 1983-2003
> >>>>>> Sun Microsystems, Inc. All rights reserved.
> >>>>>> Use is subject to license terms.
> >>>>>> configuring IPv4 interfaces: eri0.
> >>>>>> Hostname: mondeo
> >>>>>> NOTICE: /: unexpected free inode 140873, run fsck(1M) -o f
> >>>>>> NOTICE: /: unexpected free inode 140873, run fsck(1M) -o f
> >>>>>> NOTICE: /: unexpected free inode 140876, run fsck(1M) -o f
> >>>>>> NOTICE: /: unexpected free inode 140876, run fsck(1M) -o f
> >>>>>> NOTICE: /: unexpected free inode 140877, run fsck(1M) -o f
> >>>>>> mount: /var/run is already mounted, swap is busy,
> >>>>>> or the allowable number of mount points has been
> >>>> exceeded The
> >>>>>> system is coming up. Please wait.
> >>>>>> NOTICE: /: unexpected free inode 140878, run fsck(1M) -o f
> >>>>>> NOTICE: /: unexpected free inode 140878, run fsck(1M) -o
> >>>> f checking
> >>>>>> ufs filesystems
> >>>>>> /dev/rdsk/c0t0d0s7: is clean.
> >>>>>> LDAP domain name is ea.mot.com
> >>>>>> starting rpc services: rpcbind keyserv done.
> >>>>>> Setting netmask of eri0 to 255.255.255.0 Setting default IPv4
> >>>>>> interface for multicast: add net 224.0/4: gateway mondeo syslog
> >>>>> service starting.
> >>>>>> Starting the Bacula File daemon
> >>>>>> The system is ready.
> >>>>>>
> >>>>>> mondeo console login:
> >>>>>>
> >>>>>>
> >>>>>> When I do a fsck I get the following messages. The first
> >>>> one in Phase
> >>>>>> 1 doesn't seem very harmfull to me. But the second one with the
> >>>>>> missing door files which get removed doesn't make me feel very
> >>>>>> confident although the system boots up without error
> >>>> messages after
> >>>>>> the fsck is finished.
> >>>>>>
> >>>>>>
> >>>>>> # fsck /dev/dsk/c0t0d0s0
> >>>>>> ** /dev/rdsk/c0t0d0s0
> >>>>>> ** Currently Mounted on /
> >>>>>> ** Phase 1 - Check Blocks and Sizes PARTIALLY ALLOCATED INODE
> >>>>>> I=137137 CLEAR? y
> >>>>>>
> >>>>>> PARTIALLY ALLOCATED INODE I=138752 CLEAR? y
> >>>>>>
> >>>>>> PARTIALLY ALLOCATED INODE I=138753 CLEAR? y
> >>>>>>
> >>>>>> PARTIALLY ALLOCATED INODE I=138755 CLEAR? y
> >>>>>>
> >>>>>> PARTIALLY ALLOCATED INODE I=138756 CLEAR? y
> >>>>>>
> >>>>>> PARTIALLY ALLOCATED INODE I=138757 CLEAR? y
> >>>>>>
> >>>>>> ** Phase 2 - Check Pathnames
> >>>>>> UNALLOCATED I=138755 OWNER=root MODE=0 SIZE=0 MTIME=Jan
> >>>> 1 01:00
> >>>>>> 1970 NAME=/etc/sysevent/syseventconfd_event_channel/reg_door
> >>>>>>
> >>>>>> REMOVE? y
> >>>>>>
> >>>>>> UNALLOCATED I=137137 OWNER=root MODE=0 SIZE=0 MTIME=Jan
> >>>> 1 01:00
> >>>>>> 1970 NAME=/dev/.devfsadm_synch_door
> >>>>>>
> >>>>>> REMOVE? y
> >>>>>>
> >>>>>> UNALLOCATED I=138756 OWNER=root MODE=0 SIZE=0 MTIME=Jan
> >>>> 1 01:00
> >>>>>> 1970 NAME=/etc/sysevent/sysevent_door
> >>>>>>
> >>>>>> REMOVE? y
> >>>>>>
> >>>>>> UNALLOCATED I=138757 OWNER=root MODE=0 SIZE=0 MTIME=Jan
> >>>> 1 01:00
> >>>>>> 1970 NAME=/etc/sysevent/piclevent_door
> >>>>>>
> >>>>>> REMOVE? y
> >>>>>>
> >>>>>> UNALLOCATED I=138752 OWNER=root MODE=0 SIZE=0 MTIME=Jan
> >>>> 1 01:00
> >>>>>> 1970 NAME=/etc/sysevent/devfsadm_event_channel/reg_door
> >>>>>>
> >>>>>> REMOVE? y
> >>>>>>
> >>>>>> UNALLOCATED I=138753 OWNER=root MODE=0 SIZE=0 MTIME=Jan
> >>>> 1 01:00
> >>>>>> 1970
> >>>>>> NAME=/etc/sysevent/devfsadm_event_channel/1
> >>>>>>
> >>>>>> REMOVE? y
> >>>>>>
> >>>>>> ** Phase 3 - Check Connectivity
> >>>>>> ** Phase 4 - Check Reference Counts
> >>>>>> ** Phase 5 - Check Cyl groups
> >>>>>> FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? y
> >>>>>>
> >>>>>> 83047 files, 1997574 used, 3038648 free (95392 frags,
> >>>> 367907 blocks,
> >>>>>> 1.9%
> >>>>>> fragmentation)
> >>>>>>
> >>>>>> ***** FILE SYSTEM WAS MODIFIED ***** #
> >>>>>>
> >>>>>>
> >>>>>> So question is why are these doors not created correctly by the
> >>>>> restore?
> >>>>>> And will these doors get created automatically by the
> >>>> system if they
> >>>>>> don't exist?
> >>>>>> What are these partially allocated inodes?
> >>>>>> I've tried it several times and it shows the same
> >>>> behaviour each time.
> >>>>>> Regards
> >>>>>> Wolfgang
> >>>>>>
> >>>>>>
> >>> ----------------------------------------------------------------------
> >>>>>> --- Take Surveys. Earn Cash. Influence the Future of IT Join
> >>>>>> SourceForge.net's Techsay panel and you'll get the chance
> >>>> to share
> >>>>>> your opinions on IT & business topics through brief
> >>>> surveys - and earn
> >>>>>> cash
> >>>>>>
> >>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEV
> >>>>>> DEV _______________________________________________
> >>>>>> Bacula-users mailing list
> >>>>>> [email protected]
> >>>>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >>>>>>
> >>>>>
> >>>> --------------------------------------------------------------
> >>>> -----------
> >>>>> Take Surveys. Earn Cash. Influence the Future of IT Join
> >>>>> SourceForge.net's Techsay panel and you'll get the
> >>>> chance to share your
> >>>>> opinions on IT & business topics through brief surveys -
> >>>> and earn cash
> >>>> http://www.techsay.com/default.php?page=join.php&p=sourceforge
> >>>> &CID=DEVDEV
> >>>>> _______________________________________________
> >>>>> Bacula-users mailing list
> >>>>> [email protected]
> >>>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >>>>>
> >>>> --------------------------------------------------------------
> >>>> -----------
> >>>> Take Surveys. Earn Cash. Influence the Future of IT Join
> >>>> SourceForge.net's Techsay panel and you'll get the chance to share
> >>>> your opinions on IT & business topics through brief surveys -
> >>> and earn cash
> >>>> http://www.techsay.com/default.php?page=join.php&p=sourceforge
> >>>> &CID=DEVDEV
> >>>> _______________________________________________
> >>>> Bacula-users mailing list
> >>>> [email protected]
> >>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >>>>
> >>>
> >>> --------------------------------------------------------------
> >>> -----------
> >>> Take Surveys. Earn Cash. Influence the Future of IT Join
> >>> SourceForge.net's Techsay panel and you'll get the chance to share
> >>> your opinions on IT & business topics through brief surveys - and earn
> >>> cash http://www.techsay.com/default.php?page=join.php&p=sourceforge
> >> &CID=DEVDEV
> >>> _______________________________________________
> >>> Bacula-users mailing list
> >>> [email protected]
> >>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >>>
> >>
> >> ------------------------------------------------------------------------
> >> -
> >> Take Surveys. Earn Cash. Influence the Future of IT Join
> >> SourceForge.net's Techsay panel and you'll get the chance to share your
> >> opinions on IT & business topics through brief surveys - and earn cash
> >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDE
> >> V
> >> _______________________________________________
> >> Bacula-users mailing list
> >> [email protected]
> >> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >>
> >> -------------------------------------------------------------------------
> >> Take Surveys. Earn Cash. Influence the Future of IT
> >> Join SourceForge.net's Techsay panel and you'll get the chance to share
your
> >> opinions on IT & business topics through brief surveys - and earn cash
> >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> >> _______________________________________________
> >> Bacula-users mailing list
> >> [email protected]
> >> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >>
> >
> > -------------------------------------------------------------------------
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share
your
> > opinions on IT & business topics through brief surveys - and earn cash
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > _______________________________________________
> > Bacula-users mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
> >
> >
>
>
> --
> Mit freundlichen Grüßen
>
> Attila Fülöp
> Netzwerkadministrator
> __________________________________________________________
> BSMO GmbH
>
> Schwedter Straße 263, D-10119 Berlin
> Postfach 140201, D-14302 Berlin
>
> Telefon +49(30) 884 293 - 914
> Telefax +49(30) 884 293 - 941
> Mobile +49(172) 384 27 29
> E-Mail mailto:[EMAIL PROTECTED]
> -----------------------------------------------------------
>
> Medizin und Gesundheit im Internet
>
> http://www.bsmo.de
> http://www.lifeline.de
> http://www.medizin-online.de
> -----------------------------------------------------------
>
>
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users
