This patch reimplement a feature that is on biblibre/master for Koha-community/master
It adds 4 parameters:
* offset = the offset of record. Say 1000 to start rebuilding at the 1000th record of your database
* length = how many records to export. Say 400 to export only 400 records
* where = add a where clause to rebuild only a given itemtype, or anything you want to filter on
Another improvement resulting from offset & length limit is the rebuild_zebra_sliced.zsh
that will be submitted in another patch.
rebuild_zebra_sliced will slice your all database in small chunks, and, if something went wrong for a given slice, will slice the slice, and repeat, until you reach a slice size of 1, showing which record is wrong in your database.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Removed mention of -l option for limiting number of items exported, as requested
by QA manager. This can be re-added in a later patch.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 1fd8c8a4de)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit 81c6e10c81)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
The scripts run with the caveat that you must specify the path to SIPconfig.xml. The followup previously attached should take care of that issue.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 43b3fb9701)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit 36cf50e27d)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This patch just update the permission of the script, that must be executable
(cherry picked from commit e8b83c665a)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit a616321929)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
- Calculates updates date based on the upper age limit defined in the patron categories.
- Allows libraries to work on all branches or only one.
- Allows libraries to specify which Adult patron category to update child categories to.
- Allows libraries to specify a single Child patron category to update to an adult category.
- Has a test mode to display what transforms would be done on the database without executing the changes.
Includes improved help, copyright statement, and uses warnings. Also incorporates Paul's suggestions regarding --help and --man, changes -fromcat and -tocat to -f and -t, and removes a redundant update to categorycode (per M. deRooy).
To test:
Create two patron categories, a child and an adult category. Make sure they
have an upper age limit.
Create or modify some patrons in multiple branches that fall into the category
of "my birthdate is less than or equal to today's date minus the upper age
limit"
1. Run the script with no flags - nothing should happen, it will suggest you try the --help flag.
2. Run the script with the --help flag - you should see the help
3. Run the script with the -f=<child category> -t=<adult category> -v -n - should show you results from all branches but take no action and tell you what its computations are.
4. Run the script with the -f=<child category> -t=<adult category> -b=<branchcode> -v -n - should show you results from your specified branch, but take no action and tell you what it's computations are.
5. Run the script with the -f=<child category> -t=<adult category> -v -b=<branchcode> - should show you the computations and tell you how many patrons were modified in your single branch. It will not show you the information on which patrons were updated.
6. Run the script with the -f=<child category> -t=<adult category> -v - should show you the computations and tell you how many patrons were modified across all branches.
7. Run the script without the -v flag, if you care what the non-verbose output looks like.
Fixed in this revision: Known limitation - if you give it an unknown tocat, it will fail with a rather ugly error.
Minor changes to the commit message to reflect new longopts (which I missed the last time)
There is more this script could do, please feel free to take it and run.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Bug 7157 : Follow up, fixing FSF address
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit fe1a642a1a)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit 14396956c9)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
use encoding(UTF-8) rather than utf-8 for stricter
encoding
Marking output as ':utf8' only flags the data as utf8
using :encoding(UTF-8) also checks it as valid utf-8
see binmode in perlfunc for more details
In accordance with the robustness principle input
filehandles have not been changed as code may make
the undocumented assumption that invalid utf-8 is present
in the imput
Fixes errors reported by t/00-testcritic.t
Where feasable some filehandles have been made lexical rather than
reusing global filehandle vars
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 263dded818)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit 960da426a8)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Currently, -v option resets Zebra log output to default system values.
This produce amount of log specified in system defaults which is usually
too low for debugging.
This change explicitly forces all Zebra log output which create much more
chatter so it triggers with verbosity level 2
Test scenario:
1. pick koha site to reindex
2. use -v -v options to rebuild_zebra.pl to see additional output
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Verified help corrections and loglevel 2 output vs. loglevel 1 output. No issues found.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 90d68d6f5c)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit 17b3c7ce91)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This patch lets cleanup_database also purge older records from the (five) import tables and the action_logs table.
Two new command line parameters are introduced: --import and --logs.
If no number of days is specified for --zebraqueue, --import or --logs, it defaults to 30 days, 60 days resp. 180 days.
I did not add a default for --sessdays, because this parameter cannot be seen separately from parameter --sessions.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Adds new parameters and code, does not change existing behaviour
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 73c5cb3277)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit c6bfb51390)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Patch makes it possible to add fields from the issues table to overdue notices.
Template used for testing:
<item>"<<biblio.title>>" by <<biblio.author>>, <<items.itemcallnumber>>, Barcode: <<items.barcode>> , Checkout date: <<issues.issuedate>>, Due date: <<issues.date_due>> Fine: <fine>GBP</fine> Checkout date from items: <<items.onloan>></item>
Possible improvements:
- Dates are not formatted according to dateformat system preference
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 940652a6c7)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit f2adf05026)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 525c7bc4af)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit 19c7059573)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This patch fixes the SQL request giving the list of borrowers
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 1be4678a57)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit a7ef90ff87)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
(cherry picked from commit 410975a7f7)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit f497930d66)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
selection of items to be listed in an overdue notice included
both limits (upper and lower). So items with an overdue equal
to a limit appeared on both notices. This patch fix this,
including lower limit and excluding upper limit for the selection.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
(cherry picked from commit d86d62adba)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit 7d9ca3f1e4)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Marc Balmer <marc@msys.ch>
http://bugs.koha-community.org/show_bug.cgi?id=7346
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit d9f7922ad7)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit a284baee17)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Adds --maxdays command line flag to longoverdue.pl to allow the user to specify
their own $endrange value. Because sometimes 366 isn't enough!
Also adds help documentation for both --quiet and --maxdays params
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Simple change, works well
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 1f5f4981c0)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
(cherry picked from commit 5133a9df21)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit a1e3174668)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This patch allow to handle properly items containing extended characters and
send valid XML records to zebraidx
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit 4ce57a102b)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit e0b029a4f5)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This is a fairly hacky solution, a counter patch would be more than
welcome
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Updated, translated and installed German po files after applying this patch.
No problems found.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
(cherry picked from commit afaf7eaffa)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Improving the cleanup_database.pl script in two aspects:
1) In some cases CGI::Session seems to place quotes around the atime and ctime
data in the a_session field. Two regexps now take this into account.
2) If the --sessdays parameter is used, the --sessions parameter is now
implicitly enabled too.
With thanks to Ian Bays and Tom Hanstra.
Signed-off-by: Ian Bays <ian.bays@ptfs-europe.com>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit a7ee2993f0)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Adds purging completed entries from need_merge_authorities table.
If you set dontmerge to ON, you need to periodically remove records.
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit c56d12c721)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Pref MergeAuthoritiesOnUpdate does not exist; should be dontmerge
(AuthoritiesMarc.pm).
Instead of folder modified_authorities, now introducing a table for this
purpose: need_merge_authorities. This eliminates several permissions and
security issues. This change applies to AuthoritiesMarc.pm and
merge_authority.pl.
POD lines added for ModAuthority. Deprecated parameter $merge removed.
Test this patch by applying the db revision first from the second patch.
August 4, 2011: Rebased.
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Thanks Marcel. It works as advertised. Both modes are functionnal
(back):
- Immediate with dontmerge=0: After modifying an authority record, its
linked biblios are immediately modified. This isn't the case in 3.4.5.
- Delayed with dontmerge=1: After modifying an authority record, its
linked biblios are not modified. But an entry is added to
need_merge_authority new table and 'merge_authorities.pl -b' script
updates biblios.
Comment: need_merge_authority, like zebraqueue, should be cleared
from time to time.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit 05d35b0ae0)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This patch fixes an issue whereby biblios with many items (often > 500) would index,
but not the biblionumber itself, resulting in search results with a) inaccurate item counts
and b) no biblionumber to use in the link to the details page. This is due to Net::Z3950::ZOOM not providing
a mechanism for specifying different connection attributes; the maximumRecordSize ZOOM connection attribute,
if not specified, defaults to 1MB, which is less than the size of a MARC record with many, many 952 fields. Since
it is unlikely we can fix Net::Z3950::ZOOM in a timely fashion, this patch aims to build a workaround on the Koha end.
This patch changes EmbedItemsInMarcBiblio to use append_fields instead of insert_ordered_fields,
so the 999$c will come before the item records. It's VERY unlikely we will encounter more than 1MB of biblio-level MARC
content, as this would break the ISO-2709 standard by a large factor.
To this end, it also moves the fix_biblio_ids portion of get_corrected_marc_record out of rebuild_zebra.pl,
and makes it a part of GetMarcBiblio (right before EmbedItemsInMarcBiblio, so the 952s still come last). fix_biblio_ids
is kept as a subroutine for the deletion portion of rebuild_zebra.pl, which still uses it.
It also uses the subroutine parameter in GetMarcBiblio to do the EmbedItemsInMarcBiblio action, rather than having
rebuild_zebra.pl perform it on the itemless record returned from GetMarcBiblio. Simpler and cleaner that way.
To verify bug issue:
1. Find a biblio with over 700 items (or enough that the resulting MARCXML is greater than 1MB)
2. search for this biblio (in a search that would return multiple results, not just this title). You should get the title in
the results list
3. attempt to click the link to this biblio's details page; the biblionumber should be blank, leading to a 404
To test solution:
1. Apply patch
2. modify the biblio slightly (click the 005 for example) and save
OR manually add the biblio to zebraqueue for reindexing
3. after rebuild_zebra.pl -z -b -x runs, use the same search as above. The title should still appear.
4. click the link, and find yourself on the biblio detail page as desired
Signed-off-by: D Ruth Bavousett <ruth@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit 4e95e94727)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Robin Sheat <robin@catalyst.net.nz>
Note: this script really needs a rewrite, but this patch does fix up the
things it's supposed to fix up.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit cfe8b6bd9a)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This will re-add any leading spaces, so formatting is not messed
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
- all examples from the bug report are fixed now
- verified system preferences are still translated
- verified xslt displays are still translated
- verified javascript alerts are still translated
- verified switching languages works
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
(cherry picked from commit 873091dcc8)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This patch solves the situation that news is in another language than
the Koha interface AND makes that themelanguage routine is always called
the same way in order to prevent mixed display.
It fixes also a bug related to language preselection based on web
browser prefered language.
September 9: Adjusted with input of Frederic Demians.
Septembre 10: Avoid circular dependency, as pointed by Chris Cormack.
Templates related functions are moved from C4::Output to C4::Templates
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Removing unused script.
Signed-off-by: Robin Sheat <robin@catalyst.net.nz>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit f9413ca2d2)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This test validate Template Toolkit (TT) Koha files.
For the time being an unique validation is done: Test if TT files
contain TT directive within HTML tag. For example:
<li[% IF
This kind of constuction MUST be avoided because it break Koha
translation process.
This patch transform also translation specific modules into C4 modules
in order to be able to use them in test case:
C4::TTPaser
C4::TmplToken
C4::TmplTokenType
This patch is a Perl adaptation of a Haskell script from Frère Sébastien
Marie.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Notes on testing:
- translate install de-DE - worked ok
- translate update de-DE > translate install de-DE - worked ok
- running the test xt/tt_valid.t - worked ok and pointed out lots of problems.
Found no problems.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit a41100a445)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit a3e5547355)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
To test:
Verify that there is a listed job for purge_suggestions.pl in the crontab.example
Bug 6478 - adding --days support and help verbiage to the cronjob.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit 1a17c0d6bd)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
This both adds a bit of a failsafe to get_raw_biblio, and prevents
records that have been deleted from being updated by the same instance
of rebuild_zebra.
Minor amendment to remove duplication of 6433
Signed-off-by: MJ Ray <mjr@phonecoop.coop>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
(cherry picked from commit 048c0dc04e)
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Paul Poulain
Chris Cormack
Liz Rea
Colin Campbell
Chris Nighswonger
Frédéric Demians
Dobrica Pavlinusic
Marcel de Rooy
Sophie Meynieux
Marc Balmer
Ian Walls
Nahuel ANGELINETTI
Srdjan Jankovic
Jonathan Druart
Owen Leonard
Julian Maurice
Jesse Weaver