Data Deduplication

[lamont]

Hi,
What would likely be the de-dupe ratio if tsm clients do archive processing 
daily (file level, no tdps) with encryption enabled?

Thanks.

+--
|This was sent by [EMAIL PROTECTED] via Backup Central.
|Forward SPAM to [EMAIL PROTECTED]
+--

Re: Another archive/expire query

[Angus Macdonald]

So I'm right to think the "dsmc delete backup" command will remove the TSM 
backup files and leave the archived copies alone?

-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of
Richard Sims
Sent: 22 January 2008 23:56
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Another archive/expire query


Use 'dsmc Delete ARchive' to remove previously archived files.

The client manual explains how to manage Archive files.

   Richard Sims

Gallair e-bost yma gynnwys gwybodaeth gyfrinachol a/neu ddeunydd hawlfraint.  
Os ydych chin meddwl eich bod wedi derbyn yr e-bost yma drwy gamgymeriad rydym 
yn ymddiheuro am hyn; peidiwch os gwelwch yn dda â datgelu, anfon ymlaen, 
printio, copïo na dosbarthu gwybodaeth yn yr e-bost yma na gweithredu mewn 
unrhyw fodd drwy ddibynnu ar ei gynnwys: gwaherddir gwneud hynnyn gyfan gwbl a 
gallai fod yn anghyfreithlon. Rhowch wybod ir anfonwr fod y neges yma wedi mynd 
ar goll cyn ei dileu.

Mae unrhyw safbwynt neu farn a gyflwynir yn eiddo ir awdur ac nid ydynt o 
anghenraid yn cynrychioli safbwynt neu farn Ymddiriedolaeth GIG Gogledd 
Orllewin Cymru.

Gallai cynnwys yr e-bost yma gael ei ddatgelu Ir cyhoedd o dan Ddeddf Rhyddid 
Gwybodaeth 2000.  Ni does modd gwarantu cyfrinachedd y neges ac unrhyw ateb

Bydd y neges yma ac unrhyw ffeiliau cysylltiedig wedi cael eu gwirio gan 
feddalwedd canfod firws cyn eu trosglwyddo.  Ond rhaid ir sawl syn derbyn wirio 
rhag firws ei hun cyn agor unrhyw ymgysylltiad.  Nid ywr Ymddiriedolaeth yn 
derbyn unrhyw gyfrifoldeb am unrhyw golled neu niwed a allai gael ei achosi gan 
firws meddalwedd.


This e-mail may contain confidential information and/or copyright material.  If 
you believe that you have received this e-mail in error please accept our 
apologies; please do not disclose, forward, print, copy or distribute 
information in this e-mail or take any action in reliance on its contents: to 
do so is strictly prohibited and may be unlawful.  Please inform the sender 
that this message has gone astray before deleting it.

Any views or opinions presented are to be understood as those of the author and 
do not necessarily represent those of the North West Wales NHS Trust.

The contents of this e-mail may be subject to public disclosure under the 
Freedom of Information Act 2000. The confidentiality of the message and any 
reply cannot be guaranteed.

This message and any attached files will have been checked with virus detection 
software before transmission.  However, recipients must carry out their own 
virus checks before opening any attachment.  The Trust accepts no liability for 
any loss or damage, which may be caused by software viruses.

Re: Another archive/expire query

[Richard Sims]

On Jan 23, 2008, at 4:28 AM, Angus Macdonald wrote:


So I'm right to think the "dsmc delete backup" command will remove
the TSM backup files and leave the archived copies alone?



Yes.

Re: TDP for SQL question

[Del Hoobler]

Paul,

If you need to restore from the new node name, you would launch
the CLI or GUI specifying the alternate options file name.
For example, in the example below:

GUI:   TDPSQL /TSMOPTFILE=DSMARCH.OPT
CLI:   TDPSQLC RESTORE dbname full /TSMOPTFILE=DSMARCH.OPT

This will tell Data Protection for SQL to connect
to the TSM Server using the alternate options file,
thus the alternate NODENAME.

Del



"ADSM: Dist Stor Manager"  wrote on 01/22/2008
11:54:12 PM:

> OK - I have done this and it is working as far as backing up the SQL
> database goes.
>
> Now what do I do if I need to restore from this new node name back onto
> the client?
>
> Do I need to do anything with the dsm.opt and dsmarch.opt files before
> starting the restore?
>
>
> > -Original Message-
> > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On
> > Behalf Of Del Hoobler
> > Sent: Saturday, 4 August 2007 2:37 AM
> > To: ADSM-L@VM.MARIST.EDU
> > Subject: Re: [ADSM-L] TDP for SQL question
> >
> > Paul,
> >
> > Typically...  I see that people will name their node
> > the same as their primary SQL node with an extension.
> > Something like:
> > Primary:   SQLSRV23_SQL
> > Archive:   SQLSRV23_SQL_ARCH
> >
> > And they will have a separate DSM.OPT file, something like
> > DSMARCH.OPT that has the archive nodename.
> >
> > Thanks,
> >
> > Del

Webclient signon error 53

[Matthew Warren]

Hi *SM'ers,


A customer has recently  upgraded client versions on AIX machines running
AIX 4.3 and 5.3.


TSM server is v5.3.2 on z/OS 1.7

After upgrading the AIX 5.3 servers to the TSM 5.4.1 or 5.5 client, trying
to logon to the client using the web interface gives;

ANS2622S  Invalid ID or Password

The same ID and password work fine for CLI operations.

Actlog shows;

ANR0480W Session 1032 for node
nodename (AIX) terminated - connection with client severed.

tsmwebcl.log gives

21/01/08   13:46:33 (dsmcad) ANS3006I Processing request for the TSM Web
Client (185.2.123.180).
21/01/08   13:46:42 (dsmagent) ANS3002I Session started for user  (TCP/IP
10.11.5.10).

the dsmerror.log shows

21/01/08   13:46:42 scSignOnAsAdmin: Error 53 receiving SignOnAsAdminResp
verb from server


All the AIX 4.3, TSM 5.1.5 clients have continued to work fine.

Error 53 I believe is this;

/* Definitions for server signon reject codes  */
/* These error codes are in the range (51 to 99) inclusive.*/
..
..
#define DSM_RC_REJECT_ID_UNKNOWN 53
..
..


In the first instance I've searched google and IBM, IBM gives PK03989,
which is ZFS related. TsmWiki nor Quickfacts has any info,  has anyone here
seen anything similar?

Thanks,

Matt.


This message and any attachments (the "message") is
intended solely for the addressees and is confidential.
If you receive this message in error, please delete it and
immediately notify the sender. Any use not in accord with
its purpose, any dissemination or disclosure, either whole
or partial, is prohibited except formal approval. The internet
can not guarantee the integrity of this message.
BNP PARIBAS (and its subsidiaries) shall (will) not
therefore be liable for the message if modified.
Do not print this message unless it is necessary,
consider the environment.

-

Ce message et toutes les pieces jointes (ci-apres le
"message") sont etablis a l'intention exclusive de ses
destinataires et sont confidentiels. Si vous recevez ce
message par erreur, merci de le detruire et d'en avertir
immediatement l'expediteur. Toute utilisation de ce
message non conforme a sa destination, toute diffusion
ou toute publication, totale ou partielle, est interdite, sauf
autorisation expresse. L'internet ne permettant pas
d'assurer l'integrite de ce message, BNP PARIBAS (et ses
filiales) decline(nt) toute responsabilite au titre de ce
message, dans l'hypothese ou il aurait ete modifie.
N'imprimez ce message que si necessaire,
pensez a l'environnement.

Re: Webclient signon error 53

[Richard Sims]

Hi, Matthew -

I haven't run into that before, but my guess would be that the TSM
client is having trouble identifying the peer, probably because the
10.x.x.x private net address is not DNS reverse-lookupable into a
host/node name.  TSM 5.2 changed IP address handling a bit, in
concert with the DNSLOOKUP option, new at that level.

   Richard Sims   at Boston University

Re: Data Deduplication

[Ben Bullock]

 As with all questions like this, the answer is "it depends".
 It depends on the make-up of your data (# of DB full dumps, % of DB
dumps to filesystem data, % of change on the client, etc)
It depends on the vendor of DeDupe you are using.

FWIW, I am about to replace a 100TB of LTO tape with a DataDomain 560
dedupe box starting next week. Once the migration from tape to disk is
complete, I will be reporting what I saw in my environment. The DD folks
are saying that the worst case scenario will be a 7X reduction (i.e.
70TB of data squeezed into a 10TB DataDomain appliance). We shall see.

Ben
 

-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
lamont
Sent: Wednesday, January 23, 2008 1:29 AM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] Data Deduplication

Hi,
What would likely be the de-dupe ratio if tsm clients do archive
processing daily (file level, no tdps) with encryption enabled?

Thanks.

+--
|This was sent by [EMAIL PROTECTED] via Backup Central.
|Forward SPAM to [EMAIL PROTECTED]
+--


The Blue Cross of Idaho Email Firewall Server made the following annotations:
--
*Confidentiality Notice: 

This E-Mail is intended only for the use of the individual
or entity to which it is addressed and may contain
information that is privileged, confidential and exempt
from disclosure under applicable law. If you have received
this communication in error, please do not distribute, and
delete the original message. 

Thank you for your compliance.

You may contact us at:
Blue Cross of Idaho 
3000 E. Pine Ave.
Meridian, Idaho 83642
1.208.345.4550

==

Re: Webclient signon error 53

[Matthew Warren]

thanks,

I'll follow that up, post how it goes.

Matt.



   Internet
   [EMAIL PROTECTED]

 To
 ADSM-L
   Sent by: ADSM-L@VM.MARIST.EDU
 cc

   23/01/2008 14:48 
Subject
 Re: [ADSM-L] Webclient signon 
error 53

 Please respond to
ADSM-L@VM.MARIST.EDU








Hi, Matthew -

I haven't run into that before, but my guess would be that the TSM
client is having trouble identifying the peer, probably because the
10.x.x.x private net address is not DNS reverse-lookupable into a
host/node name.  TSM 5.2 changed IP address handling a bit, in
concert with the DNSLOOKUP option, new at that level.

Richard Sims   at Boston University



This message and any attachments (the "message") is
intended solely for the addressees and is confidential.
If you receive this message in error, please delete it and
immediately notify the sender. Any use not in accord with
its purpose, any dissemination or disclosure, either whole
or partial, is prohibited except formal approval. The internet
can not guarantee the integrity of this message.
BNP PARIBAS (and its subsidiaries) shall (will) not
therefore be liable for the message if modified.
Do not print this message unless it is necessary,
consider the environment.

-

Ce message et toutes les pieces jointes (ci-apres le
"message") sont etablis a l'intention exclusive de ses
destinataires et sont confidentiels. Si vous recevez ce
message par erreur, merci de le detruire et d'en avertir
immediatement l'expediteur. Toute utilisation de ce
message non conforme a sa destination, toute diffusion
ou toute publication, totale ou partielle, est interdite, sauf
autorisation expresse. L'internet ne permettant pas
d'assurer l'integrite de ce message, BNP PARIBAS (et ses
filiales) decline(nt) toute responsabilite au titre de ce
message, dans l'hypothese ou il aurait ete modifie.
N'imprimez ce message que si necessaire,
pensez a l'environnement.

Re: Data Deduplication

[Roger Deschner]

Encryption might have a DRAMATIC effect, completely eliminating the
benefits of either deduplication or compression. I predict 1:1. i.e. NO
savings for dedupliaction, with TSM client encryption.

This is why encryption at the tape drive is a very popular option with
LTO4. You can both encrypt and compress at the same time.

Roger Deschner  University of Illinois at Chicago [EMAIL PROTECTED]
   Academic Computing & Communications Center


On Wed, 23 Jan 2008, Ben Bullock wrote:

> As with all questions like this, the answer is "it depends".
> It depends on the make-up of your data (# of DB full dumps, % of DB
>dumps to filesystem data, % of change on the client, etc)
>It depends on the vendor of DeDupe you are using.
>
>FWIW, I am about to replace a 100TB of LTO tape with a DataDomain 560
>dedupe box starting next week. Once the migration from tape to disk is
>complete, I will be reporting what I saw in my environment. The DD folks
>are saying that the worst case scenario will be a 7X reduction (i.e.
>70TB of data squeezed into a 10TB DataDomain appliance). We shall see.
>
>Ben
>
>
>-Original Message-
>From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
>lamont
>Sent: Wednesday, January 23, 2008 1:29 AM
>To: ADSM-L@VM.MARIST.EDU
>Subject: [ADSM-L] Data Deduplication
>
>Hi,
>What would likely be the de-dupe ratio if tsm clients do archive
>processing daily (file level, no tdps) with encryption enabled?
>
>Thanks.
>
>+--
>|This was sent by [EMAIL PROTECTED] via Backup Central.
>|Forward SPAM to [EMAIL PROTECTED]
>+--
>
>
>The Blue Cross of Idaho Email Firewall Server made the following annotations:
>--
>*Confidentiality Notice:
>
>This E-Mail is intended only for the use of the individual
>or entity to which it is addressed and may contain
>information that is privileged, confidential and exempt
>from disclosure under applicable law. If you have received
>this communication in error, please do not distribute, and
>delete the original message.
>
>Thank you for your compliance.
>
>You may contact us at:
>Blue Cross of Idaho
>3000 E. Pine Ave.
>Meridian, Idaho 83642
>1.208.345.4550
>
>==
>

Re: Data Deduplication

[Wanda Prather]

Oooh, what a great question!
I'd guess if client encryption is on and working, the dedup ratio should be
about 1:1; because the data should never encrypt the same way twice.


On 1/23/08, lamont <[EMAIL PROTECTED]> wrote:
>
> Hi,
> What would likely be the de-dupe ratio if tsm clients do archive
> processing daily (file level, no tdps) with encryption enabled?
>
> Thanks.
>
> +--
> |This was sent by [EMAIL PROTECTED] via Backup Central.
> |Forward SPAM to [EMAIL PROTECTED]
> +--
>

Re: Data Deduplication[-UNSECURE-]

[Ben Bullock]

You are right, if client compression is turned on, you will get next to
no compression.

In the DataDomain best practices for TSM documentation, they say
to have client compression turned off for the DD appliance to do its
thing.

We don't have client compression turned on because we hate the
hit it takes on the clients and we have sufficiently big network pipes.
So we are set up OK for their appliance to work it's "magic".

Ben

-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Wanda Prather
Sent: Wednesday, January 23, 2008 8:42 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Data Deduplication

Oooh, what a great question!
I'd guess if client encryption is on and working, the dedup ratio should
be about 1:1; because the data should never encrypt the same way twice.


On 1/23/08, lamont <[EMAIL PROTECTED]> wrote:
>
> Hi,
> What would likely be the de-dupe ratio if tsm clients do archive 
> processing daily (file level, no tdps) with encryption enabled?
>
> Thanks.
>
> +-
> +-
> |This was sent by [EMAIL PROTECTED] via Backup Central.
> |Forward SPAM to [EMAIL PROTECTED]
> +-
> +-
>


The Blue Cross of Idaho Email Firewall Server made the following annotations:
--
*Confidentiality Notice: 

This E-Mail is intended only for the use of the individual
or entity to which it is addressed and may contain
information that is privileged, confidential and exempt
from disclosure under applicable law. If you have received
this communication in error, please do not distribute, and
delete the original message. 

Thank you for your compliance.

You may contact us at:
Blue Cross of Idaho 
3000 E. Pine Ave.
Meridian, Idaho 83642
1.208.345.4550

==

Re: Data Deduplication

[Wanda Prather]

I agree about client encryption wrecking dedup ratios.

FWIW however, if you turn on both COMPRESSION and ENCRYPTION on the client,
the client is also smart enough to compress first, then encrypt, so you get
the compression benefits.

However, that of course takes a lot of cycles on the client, and can really
slow down restores.  Outboard compression/encryption in the hardware is
definitely superior.


On 1/23/08, Roger Deschner <[EMAIL PROTECTED]> wrote:
>
> Encryption might have a DRAMATIC effect, completely eliminating the
> benefits of either deduplication or compression. I predict 1:1. i.e. NO
> savings for dedupliaction, with TSM client encryption.
>
> This is why encryption at the tape drive is a very popular option with
> LTO4. You can both encrypt and compress at the same time.
>
> Roger Deschner  University of Illinois at Chicago [EMAIL PROTECTED]
>   Academic Computing & Communications Center
>
>
> On Wed, 23 Jan 2008, Ben Bullock wrote:
>
> > As with all questions like this, the answer is "it depends".
> > It depends on the make-up of your data (# of DB full dumps, % of DB
> >dumps to filesystem data, % of change on the client, etc)
> >It depends on the vendor of DeDupe you are using.
> >
> >FWIW, I am about to replace a 100TB of LTO tape with a DataDomain 560
> >dedupe box starting next week. Once the migration from tape to disk is
> >complete, I will be reporting what I saw in my environment. The DD folks
> >are saying that the worst case scenario will be a 7X reduction (i.e.
> >70TB of data squeezed into a 10TB DataDomain appliance). We shall see.
> >
> >Ben
> >
> >
> >-Original Message-
> >From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
> >lamont
> >Sent: Wednesday, January 23, 2008 1:29 AM
> >To: ADSM-L@VM.MARIST.EDU
> >Subject: [ADSM-L] Data Deduplication
> >
> >Hi,
> >What would likely be the de-dupe ratio if tsm clients do archive
> >processing daily (file level, no tdps) with encryption enabled?
> >
> >Thanks.
> >
> >+--
> >|This was sent by [EMAIL PROTECTED] via Backup Central.
> >|Forward SPAM to [EMAIL PROTECTED]
> >+--
> >
> >
> >The Blue Cross of Idaho Email Firewall Server made the following
> annotations:
>
> >--
> >*Confidentiality Notice:
> >
> >This E-Mail is intended only for the use of the individual
> >or entity to which it is addressed and may contain
> >information that is privileged, confidential and exempt
> >from disclosure under applicable law. If you have received
> >this communication in error, please do not distribute, and
> >delete the original message.
> >
> >Thank you for your compliance.
> >
> >You may contact us at:
> >Blue Cross of Idaho
> >3000 E. Pine Ave.
> >Meridian, Idaho 83642
> >1.208.345.4550
> >
>
> >==
> >
>

Re: Upgrading TSM on an AIX server running multiple TSM instances

[Orville Lantto]

The reason to have separate binaries for each instance is that the process in 
AIX is tied to the binary file and holds it open.
 
Orville L. Lantto



From: ADSM: Dist Stor Manager on behalf of Loon, E.J. van - SPLXM
Sent: Tue 1/22/2008 02:55
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Upgrading TSM on an AIX server running multiple TSM 
instances



Hi Orville!
Thank you very much for you help!
I read the technote (1052631) Richard pointed out. That doesn't mention
copying the dsmserv executable.
Any reason why you choose to do so?
Kindest regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Orville Lantto
Sent: maandag 21 januari 2008 19:11
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Upgrading TSM on an AIX server running multiple TSM
instances

You do not really need a full install of the TSM server code in both
locations.  Just a separate directory, separate config files, a separate
copy of the dsmserv binary and a the appropriate environment to run it
in.

export DSMSERV_DIR=/usr/tivoli/tsm/server/bin
export DSMSERV_CONFIG=/usr/tivoli/tsm/server/instance2/dsmserv.opt

Our upgrade procedure is to do the primary install and copy the dsmserv
binary to each instance directory.  Done.


Orville L. Lantto



From: ADSM: Dist Stor Manager on behalf of Loon, E.J. van - SPLXM
Sent: Mon 1/21/2008 06:11
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] Upgrading TSM on an AIX server running multiple TSM
instances



Hi *SM-ers!
I'm running two TSM server instances on an AIX host.
Upgrading them from 5.3 to 5.4.0.0 works fine. I installed the 5.4.0.0
code through smitty, changed the /usr/tivoli/tsm/server path to the
second instance and install the 5.4.0.0 code here, using the force (-F)
flag. Both instances are on 5.4.0.0 at that point.
Now I'm trying the same trick for the upgrade to 5.4.2.0. Upgrading the
first instance works fine, but since this is an update, the -F flage is
not allowed by smitty during the upgrade of the second instance:

Force Apply Failures

The following is a list of fileset updates.  Updates cannot be specified
from the command line when the force flag (-F) is used in combination
with the apply flag (-a).

Maybe an AIX expert (and I now there are plenty of them on this list)
can help me with this one?
Thank you VERY much in advance
Kindest regards,
Eric van Loon
KLM Royal Dutch Airlines


**
For information, services and offers, please visit our web site:
http://www.klm.com    . This e-mail 
and any
attachment may contain confidential and privileged material intended for
the addressee only. If you are not the addressee, you are notified that
no part of the e-mail or any attachment may be disclosed, copied or
distributed, and that any other action related to this e-mail or
attachment is strictly prohibited, and may be unlawful. If you have
received this e-mail by error, please notify the sender immediately by
return e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or
its employees shall not be liable for the incorrect or incomplete
transmission of this e-mail or any attachments, nor responsible for any
delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch
Airlines) is registered in Amstelveen, The Netherlands, with registered
number 33014286
**
**
For information, services and offers, please visit our web site:
http://www.klm.com  . This e-mail and any attachment may 
contain
confidential and privileged material intended for the addressee
only. If you are not the addressee, you are notified that no part
of the e-mail or any attachment may be disclosed, copied or
distributed, and that any other action related to this e-mail or
attachment is strictly prohibited, and may be unlawful. If you have
received this e-mail by error, please notify the sender immediately
by return e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries
and/or its employees shall not be liable for the incorrect or
incomplete transmission of this e-mail or any attachments, nor
responsible for any delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal
Dutch Airlines) is registered in Amstelveen, The Netherlands, with
registered number 33014286
**

Re: Data Deduplication

[Hart, Charles A]

True as well as any files that are already "Compressed" We have SQL DB's
doing Flat File Dumps to Disk with compression and we see 1.7:1 Ick.
Also TDP RMAN backups can use Files per set function which if set to
more than 1 RMAN will  "multiplex" each file set differently so you see
different data every time.  We have our RMAN set to files per set =1
then the DBA's run multiple channels so we see 20:1 of course our DBA's
do fulls daily  

We've even forced Compress = No in a Server Side Client option set,
which only applies to File System backups, the compression statement
does not apply to the TDP's as far as I know.

Also do what you can to have Like Data go to the same dedupe devices
(assuming you have more than one).  Example Oracle Prod / Non-Prod with
their associated OS's go to the Same Dedupe stgpoool, Exchange etc... 

Data DeDupe can be cool, but if you do not pay attention your data types
you can ruin a good thing.  

I Cant wait to see how the newer Dedupe engines that are coming out that
perform the DeDupe process "Out Of Band" compares to the Inband DeDupe
methodology.  Of course the Inbound Devices dedupes as data comes in
which can affect Backup Performance, (just add more widgets) but it will
be interesting to see how "Out of Band" dedupe methodology will perform
if you "get behind" (i.e. Days one Backup Data is still being DeDuped
while your are taking in Day 2's Backup data, then you add in Backup
Stgpool, Reclamation etc that will force the dedupe engine to re-dupe /
re-factor the data everytime the data is read.

There's been many Dedupe Threads in this user list, you could almost
write a VTL - DeDupe Best Practice Guide.



-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Wanda Prather
Sent: Wednesday, January 23, 2008 9:42 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Data Deduplication

Oooh, what a great question!
I'd guess if client encryption is on and working, the dedup ratio should
be about 1:1; because the data should never encrypt the same way twice.


On 1/23/08, lamont <[EMAIL PROTECTED]> wrote:
>
> Hi,
> What would likely be the de-dupe ratio if tsm clients do archive 
> processing daily (file level, no tdps) with encryption enabled?
>
> Thanks.
>
> +-
> +-
> |This was sent by [EMAIL PROTECTED] via Backup Central.
> |Forward SPAM to [EMAIL PROTECTED]
> +-
> +-
>


This e-mail, including attachments, may include confidential and/or 
proprietary information, and may be used only by the person or entity to 
which it is addressed. If the reader of this e-mail is not the intended 
recipient or his or her authorized agent, the reader is hereby notified 
that any dissemination, distribution or copying of this e-mail is 
prohibited. If you have received this e-mail in error, please notify the 
sender by replying to this message and delete this e-mail immediately.

Re: Data Deduplication

[Matthew Warren]

Hmm, I was going to say I'd expect almost none, because the eencryption
wouldn't generate the same data each time through.

But maybe It depends on encyption scheme, on how keys are managed (I would
expect the same data to encrypt the same way if the same keys are used -
although I am no cryptologist), on the level at which the data is
'collated' - changed block, whole files, etc.. etc.. and how the de-dupe
algorithm of choice does it's thing.


Matt.



   Internet
   [EMAIL PROTECTED]

  To
 ADSM-L
   Sent by: ADSM-L@VM.MARIST.EDU
  cc

   23/01/2008 15:47 
 Subject
 Re: [ADSM-L] Data Deduplication

 Please respond to
ADSM-L@VM.MARIST.EDU










> As with all questions like this, the answer is "it depends".
> It depends on the make-up of your data (# of DB full dumps, % of DB
>dumps to filesystem data, % of change on the client, etc)
>It depends on the vendor of DeDupe you are using.
>
>FWIW, I am about to replace a 100TB of LTO tape with a DataDomain 560
>dedupe box starting next week. Once the migration from tape to disk is
>complete, I will be reporting what I saw in my environment. The DD folks
>are saying that the worst case scenario will be a 7X reduction (i.e.
>70TB of data squeezed into a 10TB DataDomain appliance). We shall see.
>
>Ben
>
>
>-Original Message-
>From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
>lamont
>Sent: Wednesday, January 23, 2008 1:29 AM
>To: ADSM-L@VM.MARIST.EDU
>Subject: [ADSM-L] Data Deduplication
>
>Hi,
>What would likely be the de-dupe ratio if tsm clients do archive
>processing daily (file level, no tdps) with encryption enabled?
>
>Thanks.
>
>+--
>|This was sent by [EMAIL PROTECTED] via Backup Central.
>|Forward SPAM to [EMAIL PROTECTED]
>+--
>
>
>The Blue Cross of Idaho Email Firewall Server made the following
annotations:
>--

>*Confidentiality Notice:
>
>This E-Mail is intended only for the use of the individual
>or entity to which it is addressed and may contain
>information that is privileged, confidential and exempt
>from disclosure under applicable law. If you have received
>this communication in error, please do not distribute, and
>delete the original message.
>
>Thank you for your compliance.
>
>You may contact us at:
>Blue Cross of Idaho
>3000 E. Pine Ave.
>Meridian, Idaho 83642
>1.208.345.4550
>
>==

>



This message and any attachments (the "message") is
intended solely for the addressees and is confidential.
If you receive this message in error, please delete it and
immediately notify the sender. Any use not in accord with
its purpose, any dissemination or disclosure, either whole
or partial, is prohibited except formal approval. The internet
can not guarantee the integrity of this message.
BNP PARIBAS (and its subsidiaries) shall (will) not
therefore be liable for the message if modified.
Do not print this message unless it is necessary,
consider the environment.

-

Ce message et toutes les pieces jointes (ci-apres le
"message") sont etablis a l'intention exclusive de ses
destinataires et sont confidentiels. Si vous recevez ce
message par erreur, merci de le detruire et d'en avertir
immediatement l'expediteur. Toute utilisation de ce
message non conforme a sa destination, toute diffusion
ou toute publication, totale ou partielle, est interdite, sauf
autorisation expresse. L'internet ne permettant pas
d'assurer l'integrite de ce message, BNP PARIBAS (et ses
filiales) decline(nt) toute responsabilite au titre de ce
message, dans l'hypothese ou il aurait ete modifie.
N'imprimez ce message que si necessaire,
pensez a l'environnement.

Re: Data Deduplication

[Curtis Preston]

The other posters are correct.  You will get 1:1.   Dedupe works by
finding patterns.  There are no patterns in encrypted data.

One question would be why would you do that?  Most people are encrypting
data as it leaves their site.  The best way to do that is hardware
encryption (tape drive or SAN-based).  Do that on the other side of your
dedupe box and before it goes to tape -- not at the client -- and you'll
have no issues with dedupe.

---
W. Curtis Preston
Backup Blog @ www.backupcentral.com
VP Data Protection, GlassHouse Technologies 

-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
lamont
Sent: Wednesday, January 23, 2008 12:29 AM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] Data Deduplication

Hi,
What would likely be the de-dupe ratio if tsm clients do archive
processing daily (file level, no tdps) with encryption enabled?

Thanks.

+--
|This was sent by [EMAIL PROTECTED] via Backup Central.
|Forward SPAM to [EMAIL PROTECTED]
+--

Re: Upgrading TSM on an AIX server running multiple TSM instances

[James R Owen]

Orville,
Can you clarify exactly what is the problem with multiple TSM service instances 
running from a  shared dsmserv binary?  AFAIK, IBM/Tivoli does not suggest to 
copy and run a separate executable for each TSM instance.  We have not seen any 
problems sharing the same executable among several TSM instances.

Do you use this technique to allow production TSM services to continue running 
while you upgrade the default installation TSM service?

Have you used this technique to safely run different maintenance levels of TSM 
service simultaneously on the same AIX host?  If so, have you experienced any 
problems doing this?  Did you ever need to copy additional files to the running 
directories for the other TSM services?

I'm also looking for advice: how best to make an inoperative AUTOSRVR entry in 
/etc/inittab?  We leave the tiny default TSM service in the installation 
directory for upgrade processing, and never want it to start up automatically, 
but the upgrade process recreates the entry if it has been removed.
--
[EMAIL PROTECTED]   (203.432.6693)

Orville Lantto wrote:

The reason to have separate binaries for each instance is that the process in 
AIX is tied to the binary file and holds it open.

Orville L. Lantto



From: ADSM: Dist Stor Manager on behalf of Loon, E.J. van - SPLXM
Sent: Tue 1/22/2008 02:55
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Upgrading TSM on an AIX server running multiple TSM 
instances



Hi Orville!
Thank you very much for you help!
I read the technote (1052631) Richard pointed out. That doesn't mention
copying the dsmserv executable.
Any reason why you choose to do so?
Kindest regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Orville Lantto
Sent: maandag 21 januari 2008 19:11
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Upgrading TSM on an AIX server running multiple TSM
instances

You do not really need a full install of the TSM server code in both
locations.  Just a separate directory, separate config files, a separate
copy of the dsmserv binary and a the appropriate environment to run it
in.

export DSMSERV_DIR=/usr/tivoli/tsm/server/bin
export DSMSERV_CONFIG=/usr/tivoli/tsm/server/instance2/dsmserv.opt

Our upgrade procedure is to do the primary install and copy the dsmserv
binary to each instance directory.  Done.


Orville L. Lantto



From: ADSM: Dist Stor Manager on behalf of Loon, E.J. van - SPLXM
Sent: Mon 1/21/2008 06:11
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] Upgrading TSM on an AIX server running multiple TSM
instances



Hi *SM-ers!
I'm running two TSM server instances on an AIX host.
Upgrading them from 5.3 to 5.4.0.0 works fine. I installed the 5.4.0.0
code through smitty, changed the /usr/tivoli/tsm/server path to the
second instance and install the 5.4.0.0 code here, using the force (-F)
flag. Both instances are on 5.4.0.0 at that point.
Now I'm trying the same trick for the upgrade to 5.4.2.0. Upgrading the
first instance works fine, but since this is an update, the -F flage is
not allowed by smitty during the upgrade of the second instance:

Force Apply Failures

The following is a list of fileset updates.  Updates cannot be specified
from the command line when the force flag (-F) is used in combination
with the apply flag (-a).

Maybe an AIX expert (and I now there are plenty of them on this list)
can help me with this one?
Thank you VERY much in advance
Kindest regards,
Eric van Loon
KLM Royal Dutch Airlines


**
For information, services and offers, please visit our web site:
http://www.klm.com    . This e-mail 
and any
attachment may contain confidential and privileged material intended for
the addressee only. If you are not the addressee, you are notified that
no part of the e-mail or any attachment may be disclosed, copied or
distributed, and that any other action related to this e-mail or
attachment is strictly prohibited, and may be unlawful. If you have
received this e-mail by error, please notify the sender immediately by
return e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or
its employees shall not be liable for the incorrect or incomplete
transmission of this e-mail or any attachments, nor responsible for any
delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch
Airlines) is registered in Amstelveen, The Netherlands, with registered
number 33014286
**
**
For information, services and offers, please visit our web site:
http://www.klm.com  . This e-mail and any attachment may 
contain
confidential and pr

New to List and Question

[Howard Coles]

I've been on ADSM.ORG for some time, just not on the list. 

I have an interesting query:
We're going to be upgrading to TSM 5.5 (hopefully unless serious
problems are revealed) in the near future, and then adding a secondary
server to offload some of the backups, restores, and admin processes
etc.

My question is what is the best way to get the clients, and their data
over to the new secondary server when both servers will be sharing a
Library. 
The config as I'm mulling it over now is:
Server 1, 10 LTO3's
Server 2, 8 LTO2's.

I'll leave it at that for now, and see what comes up.



See Ya'
Howard Coles Jr.
Sr. Systems Engineer
Ardent Health Services
Nashville, TN
John 3:16!

Re: Upgrading TSM on an AIX server running multiple TSM instances

[Howard Coles]

 
-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
James R Owen
Sent: Wednesday, January 23, 2008 1:18 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Upgrading TSM on an AIX server running multiple
TSM instances

Orville,
Can you clarify exactly what is the problem with multiple TSM service
instances running from a  shared dsmserv binary?  AFAIK, IBM/Tivoli does
not suggest to copy and run a separate executable for each TSM instance.
We have not seen any problems sharing the same executable among several
TSM instances.

Do you use this technique to allow production TSM services to continue
running while you upgrade the default installation TSM service?

Have you used this technique to safely run different maintenance levels
of TSM service simultaneously on the same AIX host?  If so, have you
experienced any problems doing this?  Did you ever need to copy
additional files to the running directories for the other TSM services?

I'm also looking for advice: how best to make an inoperative AUTOSRVR
entry in /etc/inittab?  We leave the tiny default TSM service in the
installation directory for upgrade processing, and never want it to
start up automatically, but the upgrade process recreates the entry if
it has been removed.
--
[EMAIL PROTECTED]   (203.432.6693)

Orville Lantto wrote:
> The reason to have separate binaries for each instance is that the
process in AIX is tied to the binary file and holds it open.
>
> Orville L. Lantto
>
> 
>
> From: ADSM: Dist Stor Manager on behalf of Loon, E.J. van - SPLXM
> Sent: Tue 1/22/2008 02:55
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] Upgrading TSM on an AIX server running multiple 
> TSM instances
>
>
>
> Hi Orville!
> Thank you very much for you help!
> I read the technote (1052631) Richard pointed out. That doesn't 
> mention copying the dsmserv executable.
> Any reason why you choose to do so?
> Kindest regards,
> Eric van Loon
> KLM Royal Dutch Airlines
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf 
> Of Orville Lantto
> Sent: maandag 21 januari 2008 19:11
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: Upgrading TSM on an AIX server running multiple TSM 
> instances
>
> You do not really need a full install of the TSM server code in both 
> locations.  Just a separate directory, separate config files, a 
> separate copy of the dsmserv binary and a the appropriate environment 
> to run it in.
>
> export DSMSERV_DIR=/usr/tivoli/tsm/server/bin
> export DSMSERV_CONFIG=/usr/tivoli/tsm/server/instance2/dsmserv.opt
>
> Our upgrade procedure is to do the primary install and copy the 
> dsmserv binary to each instance directory.  Done.
>
>
> Orville L. Lantto
>
>
>
> Hi *SM-ers!
> I'm running two TSM server instances on an AIX host.
> Upgrading them from 5.3 to 5.4.0.0 works fine. I installed the 5.4.0.0

> code through smitty, changed the /usr/tivoli/tsm/server path to the 
> second instance and install the 5.4.0.0 code here, using the force 
> (-F) flag. Both instances are on 5.4.0.0 at that point.
> Now I'm trying the same trick for the upgrade to 5.4.2.0. Upgrading 
> the first instance works fine, but since this is an update, the -F 
> flage is not allowed by smitty during the upgrade of the second
instance:
>
> Force Apply Failures
> 
> The following is a list of fileset updates.  Updates cannot be 
> specified from the command line when the force flag (-F) is used in 
> combination with the apply flag (-a).
>
> Maybe an AIX expert (and I now there are plenty of them on this list) 
> can help me with this one?
> Thank you VERY much in advance
> Kindest regards,
> Eric van Loon
> KLM Royal Dutch Airlines
>

I'm with Jim Owen above, I don't think you need completely separate
binaries, unless You WANT to keep the two at different versions.  But,
something inside me vaguely remembers reading about running two
different versions on 1 box and having problems.  

Anyway, all you have to do is make sure you run the dbupgrade for each
instance before you start up the server for all of them.  

See Ya'
Howard

Seeking thoughts/experiences on backing up large amounts (say 50 Petabytes) of data

[Bob Talda]

Folks:
  Our group has been approached by a customer who asked if we could 
backup/archive 50 petabytes of data.  And yes, they are serious.

  We've begun building questions for the customer, but as this is roughly 1000 
times the current amount of data we backup, we are
on unfamiliar turf here.

 At a high level, here are some of the questions we are asking:
1) Is the 50 Petabytes an initial, or envisioned data size?  If envisioned, how 
big is the initial data load and how fast will it grow?
2) What makes up the data: databases, video/audio files, other?   (subtext: how 
many objects are involved?  What are the
opportunities to compress/deduplicate?)
3) how is the data distributed - over a number of systems or from a 
supercluster?
4) Is the data static, or changing slowly or changing rapidly? (subtext: is it 
a backup or archive scenario)
5) What are the security requirments?
6) What are the restore (aka RTO) requirements?

  We are planning on approaching vendors to get some sense of the probable data 
center requirements (cooling, power, footprint).

  If anyone in the community has experience with managing petatybes of backup 
data, we'd appreciate any feedback we could incorporate.

  Thanks in advance!

Re: New to List and Question

[Allen S. Rout]

>> On Wed, 23 Jan 2008 13:36:55 -0600, Howard Coles <[EMAIL PROTECTED]> said:

> I have an interesting query:
> We're going to be upgrading to TSM 5.5 (hopefully unless serious
> problems are revealed) in the near future, and then adding a secondary
> server to offload some of the backups, restores, and admin processes
> etc.

> My question is what is the best way to get the clients, and their data
> over to the new secondary server when both servers will be sharing a
> Library.
> The config as I'm mulling it over now is:
> Server 1, 10 LTO3's
> Server 2, 8 LTO2's.

> I'll leave it at that for now, and see what comes up.

At the risk of sounding like a parrot:

http://open-systems.ufl.edu/services/NSAM/whitepapers/50ways.html

Was a compilation of detailed methods I came up with, once, when I was
trying to list all of them.

I haven't gotten any suggestions of "You missed " for some time,
but that may just be because I'm being ignored. ;)


- Allen S. Rout

Re: Re: Upgrading TSM on an AIX server running multiple TSM instances

[David Bronder]

James R Owen wrote:
>
> I'm also looking for advice: how best to make an inoperative AUTOSRVR
> entry in /etc/inittab?  We leave the tiny default TSM service in the
> installation directory for upgrade processing, and never want it to
> start up automatically, but the upgrade process recreates the entry if
> it has been removed.

Change the "once" to "off" in the inittab entry.  I'm not sure if the
installation/upgrade process will change it back or not, though.

But don't do this if the running dsmserv was started by that autosrvr
entry, or init will kill the running server!

(Related trick for starting the TSM server after a TSM halt without
 rebooting or manually running the dsmserv process:  change the "2"
 to "2a" in inittab, then use "telinit a" to make init start the TSM
 server the same way it would at boot.)

--
Hello World.David Bronder - Systems Admin
Segmentation Fault ITS-SPA, Univ. of Iowa
Core dumped, disk trashed, quota filled, soda warm.   [EMAIL PROTECTED]

Data Deduplication

[lamont]

Hi Curtis,
Unfortunately, this was already the case when I came, client encryption is the 
only option and the tapes are needed to be sent to offsite.
I think we need to consider this - enabling/disabling client encryption and see 
how - in the test case on the upcoming POC with a de-dupe vendor.

Thanks.


cpreston wrote:
> The other posters are correct.  You will get 1:1.   Dedupe works by
> finding patterns.  There are no patterns in encrypted data.
>
> One question would be why would you do that?  Most people are encrypting
> data as it leaves their site.  The best way to do that is hardware
> encryption (tape drive or SAN-based).  Do that on the other side of your
> dedupe box and before it goes to tape -- not at the client -- and you'll
> have no issues with dedupe.
>
> ---
> W. Curtis Preston
> Backup Blog @ www.backupcentral.com
> VP Data Protection, GlassHouse Technologies
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
> lamont
> Sent: Wednesday, January 23, 2008 12:29 AM
> To: ADSM-L@VM.MARIST.EDU
> Subject: [ADSM-L] Data Deduplication
>
> Hi,
> What would likely be the de-dupe ratio if tsm clients do archive
> processing daily (file level, no tdps) with encryption enabled?
>
> Thanks.
>
> +--
> |This was sent by [EMAIL PROTECTED] via Backup Central.
> |Forward SPAM to [EMAIL PROTECTED]
> +--


+--
|This was sent by [EMAIL PROTECTED] via Backup Central.
|Forward SPAM to [EMAIL PROTECTED]
+--

Re: Inittab and Restarting TSM instance

[Steven Harris]

Hi David

My favourite way to restart  on AIX  (ksh) is

chitab $(lsitab autosrvr | sed -e "s/once/respawn/")

and then after it has started

chitab $(lsitab autosrvr | sed -e "s/respawn/once/")


but one site I worked the guy who had set up TSM was a serious AIX nerd -
CATE and all that - and had set up the TSM Servers and multiple TSM clients
for multiple domino partitions as AIX subsystems, so you could use the AIX
startsrc, lssrc and stopsrc commands to manipulate them.  They were also in
a subsystem group so they could all be started and stopped with a single
command,  Very neat, though to make a subsystem needed a little bit of C
voodoo.

It would be nice if IBM could leverage some of its AIX expertise to add
such niceties I suppose I can dream.


Steven Harris

TSM Admin, Sydney Australia
and now AIX admin is strictly verboten...

> James R Owen wrote:
> >
> > I'm also looking for advice: how best to make an inoperative AUTOSRVR
> > entry in /etc/inittab?  We leave the tiny default TSM service in the
> > installation directory for upgrade processing, and never want it to
> > start up automatically, but the upgrade process recreates the entry if
> > it has been removed.
>
> Change the "once" to "off" in the inittab entry.  I'm not sure if the
> installation/upgrade process will change it back or not, though.
>
> But don't do this if the running dsmserv was started by that autosrvr
> entry, or init will kill the running server!
>
> (Related trick for starting the TSM server after a TSM halt without
>  rebooting or manually running the dsmserv process:  change the "2"
>  to "2a" in inittab, then use "telinit a" to make init start the TSM
>  server the same way it would at boot.)
>
> --
> Hello World.David Bronder - Systems
Admin
> Segmentation Fault ITS-SPA, Univ. of
Iowa
> Core dumped, disk trashed, quota filled, soda warm.
[EMAIL PROTECTED]