Bug 13660: Exclude export phase and use existing exported MARCXML - rebuild_zebra_sliced.sh

When looking for a bad MARC Record using the rebuild_zebra_sliced.sh, it is
useful to skip the complete MARCXML exporting from Koha and reuse the exported
files for Zebra indexing.

This patch adds a new parameter:
    -x | --exclude-export Do not export Biblios from Koha, but use the existing
                          export-dir

Which depends on the:
     -d | --export-dir     Where rebuild_zebra.pl will export data
                           Default: $EXPORTDIR

 !---------!
! TEST PLAN !
 !---------!

1. Run
     "./rebuild_zebra_sliced.sh --length 1000"
   to export 1000 MARC Records
   and slice them to one big 1000-Record chunk.
2. Realize that you get an imaginary "stack smashing detected"-error crashing
   your indexing at some Record you dont know of and can't make out from the
   indexing logging.
3. Start looking for the bad Record by running:
     "./rebuild_zebra_sliced.sh --exlude-export --chunk-size 10"
   To skip Biblios export from Koha which takes ~2h and get straight into
   splitting your exported biblios to chunks of 10, and indexing them. You
   know which chunk fails so it is much easier to find the issue there.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This commit is contained in:
Olli-Antti Kivilahti 2015-02-03 13:52:26 +02:00 committed by Jonathan Druart
parent 28dfd27a1a
commit 09e330aa24
1 changed files with 12 additions and 3 deletions

View File

@ -24,6 +24,8 @@ $scriptname -h
Default: $CHUNKSSIZE
-d | --export-dir Where rebuild_zebra.pl will export data
Default: $EXPORTDIR
-x | --exclude-export Do not export Biblios from Koha, but use the existing
export-dir
-L | --log-dir Log directory
Default: $LOGDIR
-r | --remove-logs Clean log directory before start
@ -110,6 +112,7 @@ OFFSET=0
LENGTH=
CHUNKSSIZE=10000
EXPORTDIR=/tmp/rebuild/export
EXCLUDEEXPORT=no
LOGDIR=/tmp/rebuild/logs
RMLOGS=no
NOCONFIRM=no
@ -140,6 +143,9 @@ while [ $1 ]; do
shift
LOGDIR=$1
;;
-x | --exclude-export )
EXCLUDEEXPORT=yes
;;
-r | --remove-logs )
RMLOGS=yes
;;
@ -235,6 +241,7 @@ echo "Start at offset: $OFFSET"
echo "Total number of records to index: $LENGTH"
echo "Initial chunk size: $CHUNKSSIZE"
echo "Export directory: $EXPORTDIR"
echo "Exclude re-exporting: $EXCLUDEEXPORT"
echo "Log directory: $LOGDIR"
echo "Remove logs before start? $RMLOGS"
echo "Type of record: $TYPE"
@ -274,9 +281,11 @@ if [ $RMLOGS = "yes" ]; then
rm -f $LOGDIR/*.log
fi
REBUILDZEBRA_CMD="$REBUILDZEBRA $TYPESWITCH -v -x -k -d $EXPORTDIR --offset $OFFSET --length $LENGTH --skip-index"
echo "\n$REBUILDZEBRA_CMD"
$REBUILDZEBRA_CMD
if [ $EXCLUDEEXPORT = "no" ]; then
REBUILDZEBRA_CMD="$REBUILDZEBRA $TYPESWITCH -v -x -k -d $EXPORTDIR --offset $OFFSET --length $LENGTH --skip-index"
echo "\n$REBUILDZEBRA_CMD"
$REBUILDZEBRA_CMD
fi
EXPORTFILE=
case $TYPE in