[Koha-devel] data import question & 008 use in Koha

2009-11-03 Thread Christopher Curry

Hello all,

We're in the middle of migrating MARC data to Koha.  We tried importing 
serials data into Koha and a record that had 1900+ items attached did 
not import.  I found the record and tested importing it, removing items 
until it would import and found out that 1238 was the maximum number of 
items I could import with one MARC record.  Can anyone tell me if there 
is a way to configure the import to accept more items?


I used bulkmarcimport.pl on Koha 3.0.1, running on Debian Lenny.

Also, our old ILS has a database issue that results in incomplete export 
of records created from original cataloging.  These records are missing 
008 fields.  We're working out a method of exporting the data from the 
back end and splicing it back into the MARC before import, but the data 
imports into Koha without announced or noticeable errors.  I wonder if 
we're wasting our time.  Does Koha use the 008 field for anything?


--

Cheers,

Christopher Curry
Assistant Technical Librarian / Assistant IT Officer

American Philosophical Society
105 South Fifth Street
Philadelphia, PA 19106-3386
Tel. (215) 599-4299

ccu...@amphilsoc.org 

*For technical support, please use helpd...@amphilsoc.org 
*

Main Library number: (215)440-3400
APS website: http://www.amphilsoc.org

___
Koha-devel mailing list
Koha-devel@lists.koha.org
http://lists.koha.org/mailman/listinfo/koha-devel

Re: [Koha-devel] data import question & 008 use in Koha

2009-11-03 Thread Zeno Tajoli
Hi,

>Also, our old ILS has a database issue that results in incomplete 
>export of records created from original cataloging.  These records 
>are missing 008 fields.  We're working out a method of exporting the 
>data from the back end and splicing it back into the MARC before 
>import, but the data imports into Koha without announced or 
>noticeable errors.  I wonder if we're wasting our time.  Does Koha 
>use the 008 field for anything?

I answer on the question about missing 008.
I think you can use import without 008 but you loose many indexes 
based on 008 values.
In fact 008 is an important point in a good Marc21 record.

If you use Zebra, the definition of indexes are in the file
etc/zebradb/marc_defs/marc21/biblios/record.abs
The indexes connect with 008 are:

date-entered-on-file:n:range(data,0,5),date-entered-on-file:s:range(data,0,5),
pubdate:w:range(data,7,4),pubdate:n:range(data,7,4),pubdate:y:range(data,7,4),pubdate:s:range(data,7,4),
pl:w:range(data,15,3),
ta:w:range(data,22,1),
ff8-23:w:range(data,23,1),ff8-29:w:range(data,29,1),
lf:w:range(data,33,1),
bio:w:range(data,34,1),
ln:n:range(data,35,3),
ctype:w:range(data,24,4),
Record-source:w:range(data,39,0)

So I strongly suggest you to insert records with 008.
But for test you can insert also data without.

Bye


Zeno Tajoli
CILEA - Segrate (MI)
tajoliAT_SPAM_no_prendiATcilea.it
(Indirizzo mascherato anti-spam; sostituisci quanto tra AT con @)

___
Koha-devel mailing list
Koha-devel@lists.koha.org
http://lists.koha.org/mailman/listinfo/koha-devel


Re: [Koha-devel] data import question & 008 use in Koha

2009-11-03 Thread Magnus Enger
2009/11/3 Christopher Curry :
> Hello all,
>
> We're in the middle of migrating MARC data to Koha.  We tried importing
> serials data into Koha and a record that had 1900+ items attached did not
> import.  I found the record and tested importing it, removing items until it
> would import and found out that 1238 was the maximum number of items I could
> import with one MARC record.  Can anyone tell me if there is a way to
> configure the import to accept more items?
>
> I used bulkmarcimport.pl on Koha 3.0.1, running on Debian Lenny.

I think you are running into one of the limitations that the MARC
format inhertited from the days when magnetic tape was the cool new
thing...

"Record length  (character positions 00-04 [in the leader]), contains
a five-character ASCII numeric string equal to the length of the
entire record, including itself and the record terminator. The
five-character numeric string is right justified and unused positions
contain zeroes (zero fill). ***The maximum length of a record is 9
octets***." (My attempt at emphasis.)
http://www.loc.gov/marc/specifications/specrecstruc.html#genrec

And if you have enough items you reach that limit in size.

I have this same problem with one of the records i tried to import for
my client #1 - my plan is to try and transform the record to MARCXML,
which does not impose arbitrary limits on the size of a record, and
import that with the "-m MARCXML" option for bulkmarcimport.pl.

Regards,
Magnus Enger
libriotech.no
___
Koha-devel mailing list
Koha-devel@lists.koha.org
http://lists.koha.org/mailman/listinfo/koha-devel


Re: [Koha-devel] Default Koha to MARC Mappings on nonexistent table fields?

2009-11-03 Thread Thomas Dukleth
Reply inline:


On Mon, November 2, 2009 13:30, Kyle Hall wrote:
> Hey All,
>   We have set up a plain Koha 3 install which we are using for import
> testing. I wrote a script to rebuild the biblio, biblioitems, and
> items tables from biblioitems.marcxml using the data stored in
> marc_subfield_structure. What I have found is that some of the
> framework code seems to reference table columns that do not exists.
> The columns referenced are:
>
> * bibliosubject.subject ( there is no bibliosubject table )
> * biblioitems.cn_edition
> * biblioitems.cn_prefix
>
> Can anyone provide any insight to this situation? Are these dead
> columns, or columns that were never added, or just bugs?

1.  OBSOLETE COLMN REFERENCE.

bibliosubject.subject is a legacy of subject searching of pre-Zebra Koha
in Koha 2.2 and is not currently used in non-Zebra Koha.  I had not
followed changes in the implementation of non-Zebra Koha.

References to bibliosubject.subject should be removed from the Koha MARC
frameworks.


2.  POTENTIALLY USEFUL BUT UNUSED COLUMN REFERENCES.

biblioitems.cn_edition and biblioitems.cn_prefix represent columns which
had been an important part of an attempt to enable the opportunity to
create code for managing call numbers within Koha in a more standards
compliant manner.  The columns and associated value list CN_EDITION should
have been part of Koha 3.0.  I specified them in the Koha MARC
bibliographic frameworks which I edited in consultation with Joshua
Ferraro.  He had intended to add the related columns and value lists to
the database.  Joshua had intended to have some better call number
management for Koha 3.0 but no substantial change happened.  Apparently
those particular columns were not added, although, other related call
number columns were added.

Call number management should be at the item level.  Most columns for the
call number modelled on standards compliance for holdings were moved out
of the items table to accommodate an early 3.0 development goal of storing
all the items columns within the numbers and English letters for subfields
within the single Koha MARC holdings field for indexing in Zebra.

Either the missing columns should be added to the database or other
related biblioitems columns should be removed along with references to
them in the Koha MARC frameworks.  We should not have a confusing half
measure on the issue.  The related set of columns should have uses for
appropriately making the parts of the call number available to features
corresponding to the relevant function of the particular part of the call
number but that has not happened.

Currently, it may be better to remove all the related call number parts
columns from the biblioitems table for which there is no support in the
code than to keep them in the wrong column for what should be item level
use.  Reference to such columns from the Koha MARC bibliographic
frameworks should also be removed.


3.  FUTURE DEVELOPMENT.

We should fix the problem of items needing to rely upon a single Koha MARC
holdings field to enable a better opportunity to manage call numbers well
in Koha along with many other benefits.  Strong limitations on holdings
management especially in relation to serials is one of the worst problems
of Koha.  LEK has addressed the issue but that code is not being shared. 
Tümer Garip had fixed the issue for the Near East University library in
Cyprus three years ago for which the code is available.

We should be very careful to avoid merely creating another weak foundation
in attempting to address the problem without considering the options well.
 Considering options and careful planning takes time.

Development for the Koha 3.4 release should make some progress towards
easing the reliance upon our present weak foundation.  Until we resolve
items management better for Koha, we should probably not be attempting to
store very transient items column values in the Koha MARC holdings field. 
Reducing the burden of real time reliance upon the Koha MARC holdings
field would allow an easier transition to developing a more flexible,
robust, and standards compliant model for holdings in Koha.


[...]

Thomas Dukleth
Agogme
109 E 9th Street, 3D
New York, NY  10003
USA
http://www.agogme.com
+1 212-674-3783



___
Koha-devel mailing list
Koha-devel@lists.koha.org
http://lists.koha.org/mailman/listinfo/koha-devel

Re: [Koha-devel] data import question & 008 use in Koha

2009-11-03 Thread Christopher Curry

Thanks, Magnus & Zeno, for your very useful replies.

Cheers,

Christopher Curry
Assistant Technical Librarian / Assistant IT Officer

American Philosophical Society
105 South Fifth Street
Philadelphia, PA 19106-3386
Tel. (215) 599-4299

ccu...@amphilsoc.org 

*For technical support, please use helpd...@amphilsoc.org 
*

Main Library number: (215)440-3400
APS website: http://www.amphilsoc.org



Magnus Enger wrote:

2009/11/3 Christopher Curry :
  

Hello all,

We're in the middle of migrating MARC data to Koha.  We tried importing
serials data into Koha and a record that had 1900+ items attached did not
import.  I found the record and tested importing it, removing items until it
would import and found out that 1238 was the maximum number of items I could
import with one MARC record.  Can anyone tell me if there is a way to
configure the import to accept more items?

I used bulkmarcimport.pl on Koha 3.0.1, running on Debian Lenny.



I think you are running into one of the limitations that the MARC
format inhertited from the days when magnetic tape was the cool new
thing...

"Record length  (character positions 00-04 [in the leader]), contains
a five-character ASCII numeric string equal to the length of the
entire record, including itself and the record terminator. The
five-character numeric string is right justified and unused positions
contain zeroes (zero fill). ***The maximum length of a record is 9
octets***." (My attempt at emphasis.)
http://www.loc.gov/marc/specifications/specrecstruc.html#genrec

And if you have enough items you reach that limit in size.

I have this same problem with one of the records i tried to import for
my client #1 - my plan is to try and transform the record to MARCXML,
which does not impose arbitrary limits on the size of a record, and
import that with the "-m MARCXML" option for bulkmarcimport.pl.

Regards,
Magnus Enger
libriotech.no
  
___
Koha-devel mailing list
Koha-devel@lists.koha.org
http://lists.koha.org/mailman/listinfo/koha-devel

[Koha-devel] General IRC meeting 4 November 2009

2009-11-03 Thread Galen Charlton
Hi,

Apologies for the lack of reminders.  A general IRC meeting of the
Koha project will be held tomorrow, 4 November 2009 at 19:00 UTC+0.
The meeting will be held on the #koha IRC channel at irc.katipo.co.nz.
 The home page for the meeting, which includes links to time
converters, is

http://wiki.koha.org/doku.php?id=en:events:meetings:irc_meetings:meetingnotes09nov04

The agenda for the meeting is:

1.  Update on Roadmap to 3.2.
2.  Update on Koha 3.0 Roadmap.
3.  Follow-up on actions from General IRC Meeting 7 October 2009.
4.  Agree times of next meetings.

Regards,

Galen
-- 
Galen Charlton
gmcha...@gmail.com
___
Koha-devel mailing list
Koha-devel@lists.koha.org
http://lists.koha.org/mailman/listinfo/koha-devel