This patch removes several types of strings from the
PO files that cannot be usefully translated, including
ones that consist entirely of punctuation and/or HTML entities.
Test:
1) Update PO files of some lang, xx-YY-*po
cd misc/translator
perl translate update xx-YY
2) Do it again, just in case
3) rm po/xx-YY*po~
4) Extract all msgid's, sorted
cat po/xx-YY*po | egrep "^msgid" | sort | uniq > xx-YY-pre
5) Apply the patch
6) Repeat 1-3
7) Repeat 4 again, other file
cat po/xx-YY*po | egrep "^msgid" | sort | uniq > xx-YY-post
8) Do a diff, inspect results, only strings with %s and \s
diff xx-YY-pre xx-YY-post | less
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works as described, 380 strings less to 'translate'
No koha-qa errors.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested according to test plan, works as described.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit c229553040)
DBD::Mysql provides a mysql_auto_reconnect flag. Using it avoids
the time required to do a $dbh->ping().
Benchmarks:
use Modern::Perl;
use C4::Context;
for ( 1 .. 1000 ) {
$dbh = C4::Context->dbh;
}
* without this patch on a local DB:
perl t.pl 0,49s user 0,02s system 98% cpu 0,525 total
* without this patch on a remote DB:
perl t.pl 0,52s user 0,05s system 1% cpu 37,358 total
* with this patch on a local DB:
perl t.pl 0,46s user 0,04s system 99% cpu 0,509 total
* with this patch on a remote DB:
perl t.pl 0,49s user 0,02s system 56% cpu 0,892 total
Testing the auto reconnect:
use Modern::Perl;
use C4::Context;
my $ping = $dbh->ping;
say $ping;
$dbh->disconnect;
$ping = $dbh->ping;
say $ping;
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Real improvement. No koha-qa errors
prove t/db_dependent/Circulation_issuingrules.t produces no error
prove t/db_dependent/Context.t produces no error
Test
1) dumped Koha DB, load it on a non-local server
2) run sample script whit and without patch, local and remote
use Modern::Perl;
use C4::Context;
for ( 1 .. 100000 ) {
my $dbh = C4::Context->dbh;
}
Main difference I note is with remote server
a) without patch
real 0m16.357s
user 0m2.592s
sys 0m2.132s
b) with patch
real 0m0.259s
user 0m0.240s
sys 0m0.012s
I think this could be good for DBs placed on
remote servers
Bug 10611: add a "new" parameter to C4::Context->dbh
When dbh->disconnect is called and the mysql_auto_reconnect flag is set,
the dbh is not recreated: the old one is used.
Adding a new flag, we can now force the C4::Context->dbh method to
return a new dbh.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Bug 10611: Followup: remove useless calls to dbh->disconnect
These 3 calls to disconnect are done at the end of the script, they are
useless.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 4ae3665ad8)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Conflicts:
C4/Context.pm
To test:
Set up and run the cronjobs from crontab.example with a hold set to
unsuspend today. The hold should be unsuspended.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit e6da56acb1)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Problem:
If you tell gather_print_notices.pl to write output to a location
you do not have write access to, it will silently fail to write the
data, but still mark unsent messages as sent.
Solution:
This patch adds two lines of defense:
1. Check that the location given for the output is writable
2. use "open() or die" instead of just "open()" when writing the
output
The first measure should catch most of the potential errors, but
I guess a directory can be writable, but the open() still can fail
because the disk is full or something similar.
To test:
- Make sure you have some unsent messages in the message_queue table,
that do not have an email adress
- Apply the patch
- Run the script, pointing at a location you do not have access to
write to. Check that the script exits with an appropriate error
message, and that the unsent messages are still unsent. Do this
both with and without the -s option.
- To fake passing the first line of defence, comment out line 62
and put this in instead:
if ( !$output_directory || !-d $output_directory ) {
- Run the script again as above, check you get an appropriate
error and that the message queue is not touched
- Reset line 62 to how it was
- Run the script against a directory you do have access to write to
and check that output is produced as expected and that messages
are marked as sent
- Sign off
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Works as described.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit e8a24c35f6)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patches adds support for the --test option, as well as a
short message telling the user the script is running in test mode.
Test plan :
- Launch the script with -h to see the help
- Launch the script with --test and --aut with an authtypecode
that is used in your instance
- Make sure it does the same thing as launching it with -t
- Launch the script for real and make sure it still works as
expected, deleting unused authorities.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 6657860010)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
bulkmarcimport.pl can crash when searching for duplicates if the 005
field from the incoming or local record is not defined. This patch
fixes it.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Test plan
1/ Create a record with no 005 field
2/ Try to import it checking for duplicates, notice it crashes
3/ Try with a record with a 005 field, but the one in Koha missing
one, still crashes
4/ Apply patch
5/ No more crash
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Patch fixes the problem described for importing authorities
with the bulkmarcimport.pl when trying to match with existing
records.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 28d97e3228)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
The -munge-config switch has been deprecated for years, and
trying to use it would either not work at all or, if it did "work",
almost certainly damage one's Zebra configuration for Koha.
This patch removes this switch.
To test:
[1] Run rebuild_zebra.pl and verify that no mention is made
of -munge-config.
[2] Run rebuild_zebra.pl to index records in one's test database
and verify that there are no regressions.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Removing a really dangerous option
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Ran rebuild_zebra.pl with various options and confirmed
that data was reindexed successfully.
No regressions found.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit b26870e53d)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
Resolves warning on uninitialized author in split on line 128.
Just adds the same behavior for title on line 129 for completeness.
Fixes typo on occurrences and two other minor typos.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 1219b3153e)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
On step 6 (at least), the circulation and return page redirect to the
select branch page and nothing is done.
The script should die if the user used is the sql administrator account.
Test plan:
Suppose that the sql admin account is root/root and koha/koha a
superlibrarian account.
1/
perl misc/load_testing/benchmark_staff.pl --steps=6
--url=http://admin.koha.local/cgi-bin/koha/
--password="koha" --user="koha"
should produce:
...
Step 6
...
2/
perl misc/load_testing/benchmark_staff.pl --steps=1
--url=http://admin.koha.local/cgi-bin/koha/
--password="root" --user="root"
should produce:
Authentication successful
You cannot use the database administrator account to launch this script
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 4604f595d6)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
This patch follows up on the previous patch by moving the
check for whether authority and/or biblio indexing have been
specified so that -daemon has a chance to set those modes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit b25de3e7cf)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
Based on feedback, make daemon mode imply -z -a -b and abort
on startup if flags incompatible with an incremental update daemon
are used. Update documentation to match.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 00240d6970)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
This change adds code to check the zebraqueue table with a cheap SQL query
and a daemon loop that checks for new entries and processes them incrementally
before sleeping for a controllable number of seconds. The default is 5 seconds
which provides a near realtime search index update. This is desirable particularly
for libraries that are doing active catalogue updating. The query is adjusted
based on whether -a, -b, or -a -b are specified.
Help text updated. Tested against a live 3.12 system.
Note that this fix will benefit from the fix to lack of locking (bug 11078)
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 1b0992e8d5)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
Revert commit f88f11b4f8
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Before this patch, check-url-quick.pl fails with an error. After the
patch it runs correctly.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Substitution term in overdue notices shouldn't be removed if they don't match.
It prevents use of HTML in notices to use with CSS.
Test plan :
- define an overdue notice containing some valid substitution terms + some valid html tags (<strong> or <div class="my class" to use with a css
- find a borrower with overdues that trigger this notice
- run overdue_notices.pl -html <directory>
Without patch, your html tags are missing in your html file
With the patch, they are present and can be used with a css to generate a formatted pdf file.
If you add some invalid substitution term, with the patch, they are still in the final letter, but the warn in the log helps you to correct your notice template.
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch fixes the call to BatchStageMarcRecords() made
by the comand-line record importer. At some point in the future
we could go further and teach that tool how to apply modification
templates.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch adds a more extensible and flexible debarments system to Koha. The fields
borrowers.debarred and borrowers.debarredcomment are retained for compatibility and
speed.
This system supports having debarments for multiple reasons. There are currently
three types of debarments:
OVERDUES - Generated by overdue_notices.pl if the notice should debar a patron
SUSPENSION - A punative debarment generated on checkin via an issuing rule
MANUAL - A debarment created manually by a librarian
OVERDUE debarments are cleared automatically when all overdue items have been returned,
if the new system preference AutoRemoveOverduesRestrictions is enabled. It is disabled
by default to retain current default functionality.
Whenever a borrowers debarments are modified, the system updates the borrowers debarment
fields with the highest expiration from all the borrowers debarments, and concatenates
the comments from the debarments together.
Test plan:
1) Apply patch
2) Run updatedatabase.pl
3) Verify the borrower_debarments table has been created and
populated with the pre-existing debarments
4) Run t/db_dependent/Borrower_Debarments.t
5) Manually debar a patron, with an expiration date
6) Verify the patron cannot be issued to
7) Add another manual debarment with a different expiration date
8) Verify the 'restricted' message lists the date farthest into the future
9) Add another manual debarment with no expiration date
10) Verify the borrower is now debarred indefinitely
11) Delete the indefinite debarment
12) Verify the debarment message lists an expiration date dagain
13) Enable the new system preference AutoRemoveOverduesRestrictions
14) Set an overdue notice to debar after 1 day of being overdue
15) Check out an item to a patron and backdate the due date to yesterday
16) Run overdue_notices.pl
17) Verify the OVERDUES debarment was created
18) Return the item
19) Verify the OVERDUES debarment was removed
20) Disable AutoRemoveOverduesRestrictions
21) Repeat steps 15 though 18, verify the OVERDUES debarment was *not* removed
22) Add issuing rules so that an overdue item causes a temporary debarment
23) Check out an item to a patron and backdate the due date by a year
24) Return the item
25) Verify the SUSPENSION debarment was added to the patron
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
misc/commit_file.pl now properly displays the number
of replaced and ignored items.
misc/stage_file.pl now has a new command-line switch --item-action:
--item-action action to take if --add-items is specifed;
choices are 'always_add',
'add_only_for_matches', 'add_only_for_new',
'ignore', or 'replace'
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Prior to this patch, this script would only modify the date
created and modified fields; this patch changes the behavior
so that the fields can be created if they're not already present
in the record.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Mason James <mtj@kohaaloha.com>
Bug 9192: UNIMARC_sync_date_created_with_marc_biblio.pl field creation (followup)
Add tests :
if field < 10 it must not have a subfield
if field > 9 is must have a subfield
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Both patches applied, no koha-qa errors.
Bug 9192: followup fix typo
Fix typo on usage message.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Changes only strings in the script: the help text and the
success message for updated serial issues.
Thx to Jared for proof reading!
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Cosmetic changes:
* perltidy
* q{} for query
* list of columns is query instead of SELECT *
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Your cosmetic changes missed a comma, fixed in a followup
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Script serialsUpdate.pl must be set in crontab to run daily.
It checks serials that are late, modifies status and adds a note.
This patch adds some options :
--note : defined note, a defaut one is used if not defined
--no-note : disable automatic note
--verbose : used to control output
This patch also implements confirmation option "-c" who whas present but not used.
Test plan :
Run script on a database with late serials :
- serialsUpdate.pl -h
=> you get help text in output
- serialsUpdate.pl --man
=> you get full help
- serialsUpdate.pl -v
=> you get output of changed serials but database has not changed
- serialsUpdate.pl -c -v
=> you get output of changed serials and database has changed, with note "Automatically set to late"
- serialsUpdate.pl -c
=> database has changed without anything in output
- serialsUpdate.pl -c --note "LATE"
=> database has changed, with note "LATE"
- serialsUpdate.pl -c --no-note
=> database has changed with no note
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works as described. No koha-qa errors.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
New options behave as described.
There are some existing problems with this script that I have noted
directly on the bug report.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch changes update_dbix_class_files.pl so that the connection
parameters to the reference database must be passed explicitly via
command-line switches, rather than grabbing them from the current Koha
context.
The purpose of this is to intentionally put up a roadblock to reduce
the chance that a developer (or release manager) accidentally updates
the schema files to include local testing cruft such as temporary
tables.
Usage is now as follows:
[1] Create an empty database
[2] Load the schema creation script into it, e.g.,
mysql -u dbic -pdbic dbic < installer/data/mysql/kohastructure.sql
[3] Run the schema updater:
./misc/devel/update_dbix_class_files.pl --db_user=dbic --db_name=dbic --db_passwd=dbic
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
updateDatabase.pl is a bit too close to updatedatabase.pl in installer
and may cause some confusion. I would suggest update_dbix_class_files.pl
as a unambiguous and descriptive name for this file.
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
* Added base class files for all tables in koha using
DBIx::Class::Schema::Loader.
* Added a (very basic) test file for C4::Context
* Also added dependencies in required files.
To Test:
[1] Install patch
[2] Make sure you can still connect to Koha
[3] You may optionally run this test script:
use Koha::Database;
use Data::Dumper;
my $db = Koha::Database->new();
my $schema = $db->schema();
print Dumper($schema->resultset("Borrower"));
If you run this file you should get a DBIx dump of the borrowers table.
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Test plan:
- add a fine for a patron
- verify the script does not delete this patron.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works, script do not delete a patron with fines.
No koha-qa errors with all patches applied
Test
1) Added a fine to a patron
2) run script
3) reports condition
Trying to delete patron 5... Failed to delete patron 5: patron has 10.00 in fines
4) Patron is not deleted
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Tested various combinations of options:
./delete_patrons.pl
Gives a helpful message about the use of the script.
./delete_patrons.pl -h
Outputs useful information about the use of the script
and its various options.
./delete_patrons.pl --category_code ST --library CPL
Gives the correct results in numbers and deletion was done
properly.
Also tested:
--expired_before
--not_borrowed_since
-v
Testing various conditions where a delete should not occur:
- Patron has checkouts
Patron is not in list of patrons to delete (x patrons to delete)
- Patron has fines
Patron is still in list of patrons to delete (x patrons to delete)
Checked deleted patrons had been moved to deletedborrowers.
Notes about possible enhancements:
- only print the success message 'x patrons deleted' when confirm
flag was set
- patrons with current checkouts are silently excluded from the number
of patrons to be deleted. Printing the number with current checkouts
might be helpful.
Changes made:
Fixed a small error in the documentation: expired_date is expired_before.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
[1] Patron deletion now happens atomically; if one part
of the process fails, the record isn't left in a
partially deleted state.
[2] The routine for handling lists properly during patron
deletion is now invoked.
[3] The script now prints an indication if it's run
without --confirm; otherwise, one might think that
patron records were actually being deleted.
[4] --verbose now actually does something.
Without --verbose, the script will print the dry-run
warning (if applicable), the number of patrons to be
deleted, and error messages.
With --verbose, the script will also print a line with
the borrowernumber of each patron to be deleted.
To test:
[1] Run the script with and without --verbose and compare
the, well, verbosity.
[2] Run the script without --confirm and note that the script
prints a message saying that it's running in dry-run mode.
[3] Use the script to try to delete one or more patrons that have
loans. Confirm that error messages are printed reporting
foreign constraints preventing the deletion. Also confirm that
no new rows are added to deletedborrowers for those patrons that
could not be completely deleted.
[4] Use the script to delete a patron that has a public list. Verify
that after the deletion, the public list still exists but now
has a null owner.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Pasi Kallinen
Jonathan Druart
Fridolin Somers
Bernardo Gonzalez Kriegel
Liz Rea
Fridolyn SOMERS
Magnus Enger
Gaetan Boisson
Matthias Meusburger
Galen Charlton
Marcel de Rooy
Doug Kingston
Frédéric Demians
Sophie Meynieux
Kyle M Hall
Katrin Fischer
Elliott Davis