Signed-off-by: Magnus Enger <digitalutvikling@gmail.com>
Keeps current behaviour as default.
The -append option is described in the POD and works as expected.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described.
Adding a date/time to the output might
be good, to make it easier to find the entry you were looking for.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 5935654bac)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
installer/data/mysql/sysprefs.sql has semicolon as default.
This fixes both instances to use the same fallback value.
It also prevents CSV header info from being included in non-CSV messages.
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 7408fa5d70)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This determines if the CSV header should be included or not and
then generates it as needed using the delimiter specified in the
delimiter system preference.
TEST PLAN
---------
1. make some overdues books
2. run the overdue notices script without the -csv
3. check emails notice csv header is in the email
4. apply the patch
5. run the overdue notice again
6. check email notice CSV header is absent
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit a31210efd3)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
A link to course reserves is in the global header menu but not on the
home page. All links from the global header should be present on the
home page as well. This patch adds it.
To test, apply the patch and if necessary clear your browser cache. View
the staff client home page. If you have "UseCourseReserves" enabled you
should see a link for the course reserves page which is visually
consistent with the other module links. If you do not have course
reserves enabled you should not see the link.
Unrelated: I positioned the admin link after the tools link because it
bugged me.
Signed-off-by: Broust <jean-manuel.broust@univ-lyon2.fr>
Signed-off-by: marjorie barry-vila <marjorie.barry-vila@ccsr.qc.ca>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script, works as described.
Course reserves is still accessible without permissions, but
you can't make any changes to the reserves then.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 8a9cdcd3bc)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch improves the CSS used to attempt to Zebra-stripe the
output of emailed reports. This will work with some email clients,
but other email clients (e.g., Gmail) don't handle style elements in the
body or head element.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 26cdaaae77)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
The misc/cronjobs/runreport.pl allows for sending html reports
via email. The problem is that the Content-Type isn't set to
text/html, which means that the generated html email isn't
displayed properly.
This patch set the Content-Type, and also adds a tiny bit of
CSS to potentially alternate row colours (just to make long
reports a bit easier on the eye!)
TEST PLAN
----------
1. Run the script similar to this:
./misc/cronjobs/runreport.pl --format=html --to=YOUREMAIL --subject="Bad Formatting!" REPORTNUMBER
2. Look at the email - the html code should by visible and ugly.
3. apply the patch
4. Run the script again.
5. Look at the email - the data should look nicer now.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 25076dec54)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch improves rebuild_zebra.pl's usage help
by explaining when --skip-deletes should be considered
and noting that it should be used in conjunction with
a cronjob to process deletions after hours.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 03338b70e4)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
It seems that record deletions can cause extreme slowdowns for Koha
installations with extremely large numbers of records. It would be
helpful to be able to skip record deletions when processing the
zebraqueue with rebuild_zebra.pl so the deletions can be processed with
a lower frequency.
Test Plan:
1) Disable any zebra indexing cronjobs you may have
2) Delete a record
3) Note the operation recordDelete in the zebraqueue table having done = 0
4) Run misc/migration_tools/rebuild_zebra.pl -b -z --skip-deletes
5) Note the delete still has done = 0
6) Run misc/migration_tools/rebuild_zebra.pl -b -z
7) Note the delete now has done = 1
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Also tested for authorities, no problems found.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
RM note: this is at best a work-around, and I will emphasize that
--skip-deletes should be used only when absolutely necessary.
I hope that --skip-deletes can go away at some point soon, but
that may depend on changes to Zebra.
(cherry picked from commit b0870311e1)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
- fix a couple typos in comments
- make replace a "$i" with a more descriptive variable name
- style some of the new code
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 160c44d4e9)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
The original patch creates a lockfile in the ZEBRA_LOCKDIR.
It can fall back to /var/lock or even /tmp.
If the create fails, it dies. This can be considered as very
exceptional.
This followup adjusts the fallback location in /var/lock or /tmp
slightly. It appends the database name to the folder in order to
prevent interfering between multiple Koha instances. Creation of the
lockfile has been moved to a subroutine extending directory and file
creation testing.
In the very unlikely case that we cannot create the lockfile (after
three separate tries), this follow-up allows you to continue instead
of die. This is just as we did before we had file locking here. Every
time skipping a reindex could cause more harm than continuing and
having the race condition once in a while.
Test plan:
Test adding and removing lockdir from your koha-conf.xml. Check fallback.
Note that fallback in /var/lock or /tmp must contain database name.
Remove the lockdir config line and remove permissions from fallback. In
this case the reindex should continue but with a warning.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested with daemon and one-off invocation simultaneously.
Tested new wait parameter.
Tried all variations of lock directory (changing permissions etc.)
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 07de37f0e5)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch adds locking to rebuild_zebra.pl to ensure that simultaneous
changes are prevented (as one is likely to overwrite the other).
Incremental updates in daemon mode will skipped if the lock is busy
and they will be picked up on the next pass. Non-daemon mode
invocations will also exit immediately if they cannot get the lock
unless the new flag -wait-for-lock is specified, in which case they
will wait until the get the lock and then proceed.
Supporting changes made to Makefile.PL and templates for the new
locking directory (paralleling the other zebra lock directories).
We stash the zebra_lockdir in koha-conf.xml so rebuild_zebra.pl
can find it.
To address earlier QA concerns we:
1. added code to check if flock is available and ignore locking if
it's missing (from M. de Rooy)
2. changed default for adhoc invocations to abort if they cannot
obtain the lock. Added option -wait-for-lock if the user prefers
to wait until the lock is free, and then continue processing.
3. added missing entry to t/db_dependent/zebra_config.pl
4. added a fallback locking directory of /tmp
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Doug merged the original patch with the QA changes.
Just for the record, noting here that the original patch was tested
extensively too by Martin Renvoize.
I have added a followup for some exceptional cases.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 88e7faf860)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch removes several types of strings from the
PO files that cannot be usefully translated, including
ones that consist entirely of punctuation and/or HTML entities.
Test:
1) Update PO files of some lang, xx-YY-*po
cd misc/translator
perl translate update xx-YY
2) Do it again, just in case
3) rm po/xx-YY*po~
4) Extract all msgid's, sorted
cat po/xx-YY*po | egrep "^msgid" | sort | uniq > xx-YY-pre
5) Apply the patch
6) Repeat 1-3
7) Repeat 4 again, other file
cat po/xx-YY*po | egrep "^msgid" | sort | uniq > xx-YY-post
8) Do a diff, inspect results, only strings with %s and \s
diff xx-YY-pre xx-YY-post | less
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works as described, 380 strings less to 'translate'
No koha-qa errors.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested according to test plan, works as described.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit c229553040)
DBD::Mysql provides a mysql_auto_reconnect flag. Using it avoids
the time required to do a $dbh->ping().
Benchmarks:
use Modern::Perl;
use C4::Context;
for ( 1 .. 1000 ) {
$dbh = C4::Context->dbh;
}
* without this patch on a local DB:
perl t.pl 0,49s user 0,02s system 98% cpu 0,525 total
* without this patch on a remote DB:
perl t.pl 0,52s user 0,05s system 1% cpu 37,358 total
* with this patch on a local DB:
perl t.pl 0,46s user 0,04s system 99% cpu 0,509 total
* with this patch on a remote DB:
perl t.pl 0,49s user 0,02s system 56% cpu 0,892 total
Testing the auto reconnect:
use Modern::Perl;
use C4::Context;
my $ping = $dbh->ping;
say $ping;
$dbh->disconnect;
$ping = $dbh->ping;
say $ping;
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Real improvement. No koha-qa errors
prove t/db_dependent/Circulation_issuingrules.t produces no error
prove t/db_dependent/Context.t produces no error
Test
1) dumped Koha DB, load it on a non-local server
2) run sample script whit and without patch, local and remote
use Modern::Perl;
use C4::Context;
for ( 1 .. 100000 ) {
my $dbh = C4::Context->dbh;
}
Main difference I note is with remote server
a) without patch
real 0m16.357s
user 0m2.592s
sys 0m2.132s
b) with patch
real 0m0.259s
user 0m0.240s
sys 0m0.012s
I think this could be good for DBs placed on
remote servers
Bug 10611: add a "new" parameter to C4::Context->dbh
When dbh->disconnect is called and the mysql_auto_reconnect flag is set,
the dbh is not recreated: the old one is used.
Adding a new flag, we can now force the C4::Context->dbh method to
return a new dbh.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Bug 10611: Followup: remove useless calls to dbh->disconnect
These 3 calls to disconnect are done at the end of the script, they are
useless.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 4ae3665ad8)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Conflicts:
C4/Context.pm
To test:
Set up and run the cronjobs from crontab.example with a hold set to
unsuspend today. The hold should be unsuspended.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit e6da56acb1)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Problem:
If you tell gather_print_notices.pl to write output to a location
you do not have write access to, it will silently fail to write the
data, but still mark unsent messages as sent.
Solution:
This patch adds two lines of defense:
1. Check that the location given for the output is writable
2. use "open() or die" instead of just "open()" when writing the
output
The first measure should catch most of the potential errors, but
I guess a directory can be writable, but the open() still can fail
because the disk is full or something similar.
To test:
- Make sure you have some unsent messages in the message_queue table,
that do not have an email adress
- Apply the patch
- Run the script, pointing at a location you do not have access to
write to. Check that the script exits with an appropriate error
message, and that the unsent messages are still unsent. Do this
both with and without the -s option.
- To fake passing the first line of defence, comment out line 62
and put this in instead:
if ( !$output_directory || !-d $output_directory ) {
- Run the script again as above, check you get an appropriate
error and that the message queue is not touched
- Reset line 62 to how it was
- Run the script against a directory you do have access to write to
and check that output is produced as expected and that messages
are marked as sent
- Sign off
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Works as described.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit e8a24c35f6)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patches adds support for the --test option, as well as a
short message telling the user the script is running in test mode.
Test plan :
- Launch the script with -h to see the help
- Launch the script with --test and --aut with an authtypecode
that is used in your instance
- Make sure it does the same thing as launching it with -t
- Launch the script for real and make sure it still works as
expected, deleting unused authorities.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 6657860010)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
bulkmarcimport.pl can crash when searching for duplicates if the 005
field from the incoming or local record is not defined. This patch
fixes it.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Test plan
1/ Create a record with no 005 field
2/ Try to import it checking for duplicates, notice it crashes
3/ Try with a record with a 005 field, but the one in Koha missing
one, still crashes
4/ Apply patch
5/ No more crash
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Patch fixes the problem described for importing authorities
with the bulkmarcimport.pl when trying to match with existing
records.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 28d97e3228)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
The -munge-config switch has been deprecated for years, and
trying to use it would either not work at all or, if it did "work",
almost certainly damage one's Zebra configuration for Koha.
This patch removes this switch.
To test:
[1] Run rebuild_zebra.pl and verify that no mention is made
of -munge-config.
[2] Run rebuild_zebra.pl to index records in one's test database
and verify that there are no regressions.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Removing a really dangerous option
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Ran rebuild_zebra.pl with various options and confirmed
that data was reindexed successfully.
No regressions found.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit b26870e53d)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
Resolves warning on uninitialized author in split on line 128.
Just adds the same behavior for title on line 129 for completeness.
Fixes typo on occurrences and two other minor typos.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 1219b3153e)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
On step 6 (at least), the circulation and return page redirect to the
select branch page and nothing is done.
The script should die if the user used is the sql administrator account.
Test plan:
Suppose that the sql admin account is root/root and koha/koha a
superlibrarian account.
1/
perl misc/load_testing/benchmark_staff.pl --steps=6
--url=http://admin.koha.local/cgi-bin/koha/
--password="koha" --user="koha"
should produce:
...
Step 6
...
2/
perl misc/load_testing/benchmark_staff.pl --steps=1
--url=http://admin.koha.local/cgi-bin/koha/
--password="root" --user="root"
should produce:
Authentication successful
You cannot use the database administrator account to launch this script
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 4604f595d6)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
This patch follows up on the previous patch by moving the
check for whether authority and/or biblio indexing have been
specified so that -daemon has a chance to set those modes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit b25de3e7cf)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
Based on feedback, make daemon mode imply -z -a -b and abort
on startup if flags incompatible with an incremental update daemon
are used. Update documentation to match.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 00240d6970)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
This change adds code to check the zebraqueue table with a cheap SQL query
and a daemon loop that checks for new entries and processes them incrementally
before sleeping for a controllable number of seconds. The default is 5 seconds
which provides a near realtime search index update. This is desirable particularly
for libraries that are doing active catalogue updating. The query is adjusted
based on whether -a, -b, or -a -b are specified.
Help text updated. Tested against a live 3.12 system.
Note that this fix will benefit from the fix to lack of locking (bug 11078)
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
(cherry picked from commit 1b0992e8d5)
Signed-off-by: Fridolin SOMERS <fridolin.somers@biblibre.com>
Revert commit f88f11b4f8
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Before this patch, check-url-quick.pl fails with an error. After the
patch it runs correctly.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Substitution term in overdue notices shouldn't be removed if they don't match.
It prevents use of HTML in notices to use with CSS.
Test plan :
- define an overdue notice containing some valid substitution terms + some valid html tags (<strong> or <div class="my class" to use with a css
- find a borrower with overdues that trigger this notice
- run overdue_notices.pl -html <directory>
Without patch, your html tags are missing in your html file
With the patch, they are present and can be used with a css to generate a formatted pdf file.
If you add some invalid substitution term, with the patch, they are still in the final letter, but the warn in the log helps you to correct your notice template.
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch fixes the call to BatchStageMarcRecords() made
by the comand-line record importer. At some point in the future
we could go further and teach that tool how to apply modification
templates.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Fridolin Somers
Bernardo Gonzalez Kriegel
Matthias Meusburger
Jonathan Druart
Daniel Barker
Owen Leonard
Galen Charlton
Nicholas van Oudtshoorn
Kyle M Hall
Marcel de Rooy
Doug Kingston
Pasi Kallinen
Liz Rea
Fridolyn SOMERS
Magnus Enger
Gaetan Boisson
Frédéric Demians
Sophie Meynieux