- koha-systemd.README.Debian was written for the first prototype: it
named koha-worker-long@.service (now koha-worker-long_tasks@), said
SIP needs the sip.enabled flag file (no longer read on systemd hosts),
described coexisting with koha-common and a manual migration that the
postinst now performs. Rewrite it to describe the actual units, how
the koha-* helpers map onto them, where each configuration value is
read from, how to relax the sandboxing with a drop-in, and the
migration.
- Logging is the one visible behavioural change and was undocumented:
with koha-sysv the daemon(1) wrapper captured stdout/stderr to
/var/log/koha/INSTANCE/NAME-output.log and NAME-error.log; with
koha-systemd that output is in the journal, so zebra-*, indexer-*,
es-indexer-*, worker-* and sip-error.log are no longer written, while
everything Koha writes itself (Starman's plack.log/plack-error.log,
the log4perl files, z3950.log, the Apache vhost logs) is unchanged.
Spell that out in README.Debian, in a new koha-systemd NEWS.Debian
entry shown by apt-listchanges on upgrade, and in a "Service
management" section of the koha-common(8) man page.
- Add --enable/--disable to the koha-zebra, koha-worker, koha-indexer
and koha-es-indexer man pages.
- z3950_responder.pl's synopsis listed --config-dir twice and had lost
-c, which is still a YAZ pass-through option; put -c back.
- t/00-valid-systemd-units.t: give it the usual shebang, and ignore
every "Command ... is not executable" line from systemd-analyze rather
than only the ones for scripts in this repository. Whether starman or
/usr/share/koha/bin/... is installed on the host running the test says
nothing about the unit files, and the old filter made the test fail on
any development machine without the Koha packages.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Documentation is wrong, -c doesn't work for z3950_responder.pl
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch copies the implementation from the general background
worked into the es_indexer daemon
Note: There is a 'sleep(10)' in the general indexer and I copied it here,
but I don't know why we do it?
To test:
1 - Apply the patch
2 - sudo koha-es-indexer --restart kohadev
3 - perl misc/maintenance/touch_all_biblios.pl
4 - Wait a bit
5 - Check the background jobs in the admin interface
6 - Confirm the jobs finish and records are searchable
7 - Test with and without rabbitmq enabled:
. Stop rabbitmq service: sudo -s service rabbitmq-server stop
. Repeat steps 2 to 5
8 - Sign off
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Aleisha Amohia <aleisha@catalyst.net.nz>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The mutual-exclusivity check ran before the empty-element cleanup added
for bug 37075, so a bare -c (no value) left an empty string in
@letter_code, which the check saw as truthy and rejected even though
-c had effectively not been used. Move the cleanup above the check.
Also add the same mutual-exclusivity guard directly in
C4::Letters::SendQueuedMessages, since letter_code and
exclude_letter_code build the same search key and passing both would
otherwise let one silently clobber the other in the hashref passed to
->search - only the CLI script enforced this until now, so any other
caller could hit the silent-clobber bug.
Adds regression coverage: unit tests for the new exception in
SendQueuedMessages, and a script-level test reproducing the RM's
reported false positive.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Both options were being accepted but only the last one in the command line
was being applied. Now they are mutually exclusive like --category and
--skip-category in longoverdue.pl.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Some libraries want to send digest messages once per day (e.g., at 20:05)
but send all other messages every hour. Currently, this requires listing
almost all letter codes (~85) in the hourly cron job, which is error-prone
and makes it easy to forget notifications at upgrades.
This patch adds a new --exclude-code (-x) option to process_message_queue.pl
that allows excluding specific letter codes from processing, making it much
easier to configure digest scheduling.
Changes:
- Added --exclude-code option to process_message_queue.pl (repeatable)
- Modified C4::Letters::SendQueuedMessages to handle exclude_letter_code parameter
- Supports both scalar and array reference formats
Example usage:
# Send all messages except digests every hour
00 */1 * * * process_message_queue.pl --exclude-code DUEDGST \
--exclude-code PREDUEDGST --exclude-code HOLDDGST \
--exclude-code AUTO_RENEWALS_DGST
# Send all messages including digests once per day
05 20 * * * process_message_queue.pl
Test plan:
1. Apply both patches
2. Run: ktd --shell --run 'prove t/db_dependent/Letters.t'
3. Verify all tests pass (should show 105 tests passing)
4. Create test messages in message_queue with different letter codes:
- Insert messages with codes: DUEDGST, PREDUEDGST, ACQ_NOTIF, HOLD
5. Test single exclusion:
misc/cronjobs/process_message_queue.pl --exclude-code DUEDGST
- Verify DUEDGST messages remain pending
- Verify other messages are processed
6. Test multiple exclusions:
misc/cronjobs/process_message_queue.pl --exclude-code DUEDGST \
--exclude-code PREDUEDGST
- Verify both digest messages remain pending
- Verify non-digest messages are processed
7. Test combined with --code option (should work together):
misc/cronjobs/process_message_queue.pl --code ACQ_NOTIF \
--exclude-code DUEDGST
- Verify only ACQ_NOTIF messages processed (DUEDGST excluded anyway)
8. Test help output:
misc/cronjobs/process_message_queue.pl --help
- Verify --exclude-code option is documented
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Marion Durand <marion.durand@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
- cleanup_database.pl: rewrite DeleteSpecialHolidays to purge the new
library_single_closures and library_closure_exceptions tables via their
Koha::Objects classes (was querying the dropped special_holidays table).
- kohastructure.sql: fix the comment header above library_repeating_closures
that still referred to repeatable_holidays.
- C4::Overdues: drop the dead GetSpecialHolidays / GetRepeatableHolidays /
GetWdayFromItemnumber routines (unreferenced, exported nowhere) that still
queried the dropped special_holidays / repeatable_holidays tables, along
with the now-unused Date::Calc and Date::Manip imports.
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Koha::Calendar is, by construction, a per-library object — it croaks
without a branchcode argument and every method is branch-scoped. The
top-level namespace hides that. Move the whole tree under
Koha::Library::Calendar to match the actual semantics, the existing
/libraries/{library_id}/calendar REST routes, and the established
Koha::Library::* convention (Group, Hour, FloatLimit, OverDriveInfo).
This commit:
- renames the Koha::Calendar(::*) tree to Koha::Library::Calendar(::*)
- updates package declarations, POD and internal 'use' statements
within the moved tree
- updates the four DBIC Schema Result overrides (koha_object_class /
koha_objects_class) to the new namespace
- updates callers inside C4::* modules (Calendar, Circulation,
Reserves, Overdues, HoldsQueue), Koha::{Patron,Hold,Checkouts,
CurbsidePickup}, circ/returns.pl, the installer and cronjobs
- updates the 42310-owned tests (Calendar tests, HoldsQueue.t,
Holds/WaitingReserves.t, Holidays.t, Koha/CurbsidePickups.t,
Reserves/CancelExpiredReserves.t and the libraries.t CRUD block)
Koha::Library->calendar, the REST controllers and the 42311 tests
are updated in separate follow-up commits under their respective
bugs.
Test plan:
- prove t/db_dependent/Koha/Library/Calendar.t
- prove t/db_dependent/Calendar.t
- prove t/db_dependent/Holidays.t t/db_dependent/HoldsQueue.t
- prove t/db_dependent/Holds/WaitingReserves.t
- prove t/db_dependent/Reserves/CancelExpiredReserves.t
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The following issues were identified and resolved:
- items.itemtype does not exist -> swap to items.itype
- leftover WHERE clause still referred to overduerules -> removed
- with the join on overduerules removed there is no need to GROUP BY anymore
- Koha::CirculationRules->all -> use next() instead
Test plan (as per comment 872):
A) In KTD, set 'TalkingTechItivaPhoneNotification' to 'On'.
B) Run perl misc/cronjobs/thirdparty/TalkingTech_itiva_outbound.pl --type=OVERDUE -o /tmp/out.txt -v
C) No errors / the script does not die mid run.
Quick disclaimer: I am not overly familiar with that script:
while I did remove the causes behind the script dying mid run,
I won't have spotted issues with expected output / outcome.
Assisted-by: Claude Opus 5 (Anthropic)
Sponsored-by: Glasgow Colleges Library Group <https://library.cityofglasgowcollege.ac.uk>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
- overdue_notices.pl: fix verbose sprintf - remove spurious \$triggered
argument (CLI 0/1 flag had replaced delay value in output) and use
the \$borr string that was computed but never included in the message
- overdue_notices.pl: remove %already_queued hash which was populated
in _enact_trigger but never read; dead code since the refactor
- C4/Overdues.pm GetBranchcodesWithOverdueRules: guard the global-rule
expansion with @branchcodes check so an empty result (no rules at all)
correctly returns an empty list instead of all branches
Signed-off-by: Michaela <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
- Patron.pm has_restricting_overdues: sort ASC not DESC so the oldest
(most overdue) item is checked per branch/itemtype; DESC caused the
deduplication skip to discard the very items most likely to trigger
a restriction
- overdue_notices.pl: fix --itemtypesout option name in GetOptions
(was 'itemtypeouts', never matched the documented --itemtypesout flag)
- overdue_notices.pl: fix typo 'catagorycode' -> 'categorycode' in
--borcatout search, which silently broke category exclusion
- circulation_triggers.pl: add flagsrequired => parameters/manage_circ_rules
so the admin page enforces the correct permission; also add missing
use Koha::Notice::Templates
Signed-off-by: Michaela <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
- Store patron name/address fields in \$borrower_overdues so
prepare_letter_for_printing has data for printed/CSV/text notices
- Fix email->print fallback: look up items under original MTT key,
not the effective (fallback) key
- Set \$admin_email_address from \$library unconditionally in
_enact_trigger; previously undef in non-patron_homelibrary mode,
stripping the from_address on all queued notices
Signed-off-by: Michaela <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Sponsored-by: Glasgow Colleges Library Group <https://library.cityofglasgowcollege.ac.uk>
Signed-off-by: George Harkins <George.Harkins@cityofglasgowcollege.ac.uk>
Signed-off-by: Michaela <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch enhances the overdue_notices script to understand the newly
introduced itemtype level for overdue rules.
The original logic for digesting overdues in the overdue notices script
was flawed given the additional itemtype handling introduced in this
patchset and the way in which it handles effective fallbacks for rules.
This patch tidies the file and reworks the logic so that we fetch
overdues by patron category and branch and then iterate through those
comparing to the correct effective rules and combing them into digest
groups based on notice + trigger level.
Sponsored-by: Glasgow Colleges Library Group <https://library.cityofglasgowcollege.ac.uk>
Signed-off-by: George Harkins <George.Harkins@cityofglasgowcollege.ac.uk>
Signed-off-by: Michaela <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The daemon runs as a long-lived process and never clears its L1 cache.
System preference changes (e.g. IncludeSeeFromInSearches) are not
picked up until the daemon is manually restarted.
Flush L1 caches at the start of each commit() call so that preference
changes take effect on the next indexing batch without requiring a
daemon restart.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
There was a mismatch between the documentation and the shortcut used
in the code. This patch fixes the documentation so it matches the code:
'pf|preserve-field=s' => \@preserve_fields,
=item B<-k|--preserve-field>
Prevent specified patron fields for existing patrons from being overwritten
So pf is the right shortcut.
Signed-off-by: Aleisha Amohia <aleisha@catalyst.net.nz>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The patch added cronlogaction() calls but did not import the function.
This caused a runtime error: Undefined subroutine &main::cronlogaction.
Test plan:
1. Apply patch
2. Run:
$ ktd --shell
k$ perl misc/cronjobs/cleanup_database.pl --illrequests-days 30 \
--illrequests-status COMP -v
=> SUCCESS: Dry-run works, reports count of requests that would be removed
3. Run with --confirm to actually purge
4. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This adds two new parameters:
--illrequests-status
--illrequests-days (optional, defaults to 30 days)
These can be repeated to include specific statuses for cleanup. At least one status must be specified.
To test:
0 - Apply patch
1 - Create several ILL requests with different statuses (e.g., CANCELLED, REJECTED, COMPLETED, PENDING)
2 - Set some requests to be older than 30 days (update their updated timestamp)
3 - Run cleanup_database.pl with --illrequests-status CANCELLED
4 - CANCELLED requests older than 30 days are removed, others remain
5 - Run cleanup_database.pl with --illrequests-status CANCELLED --illrequests-status REJECTED
6 - Both CANCELLED and REJECTED requests older than 30 days are removed, other statuses remain
7 - Run cleanup_database.pl with --illrequests-days 5 --illrequests-status PENDING
8 - PENDING requests older than 5 days are removed
9 - Verify dry-run mode (no --confirm) correctly counts records that would be deleted without deleting them
10 - Run cleanup_database.pl with --illrequests-days without any --illrequests-status
11 - Error message displayed requiring at least one status to be specified
Sponsored-by: Koha-Suomi Oy
Signed-off-by: John Doe <tomascohen@gmail.com>
edit: tidied the file (tcohen)
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file.
This patch fixes 3 issues
- broken (invalid) records from batch get now counted (record_number)
- when using the number param (n) to limit the number of records to be processed valid and invalid records get counted
- when using th offset param (o) the scipt errored when broken records were part of the offset.
To test:
1. Download the MARC file with the broken record
2. Run the following import commands and compare the counted import numbers:
a) all records
perl misc/migration_tools/bulkmarcimport.pl -b -v -m MARCXML --file 2_records_1st_broken.xml
1 MARC records done in 0.0421221256256104 seconds
b) limit to 1 record in total
perl misc/migration_tools/bulkmarcimport.pl -b -v -n 1 -m MARCXML --file 2_records_1st_broken.xml
1 MARC records done in 0.0420489311218262 seconds
c) limit to 1 record offset 1 record
perl misc/migration_tools/bulkmarcimport.pl -b -v -n 1 -o 1 -m MARCXML --file 2_records_1st_broken.xml
:2: parser error : Opening and ending tag mismatch: leader2 line 2 and leader
<leader2>00216nam a22001097a 4500</leader>
3. Apply the patch
4. Repeat steps in 2 and compare the results
a) 2 MARC records done in 0.0405838489532471 seconds
b) 1 MARC records done in 0.00948596000671387 seconds *
c) 1 MARC records done in 0.0294840335845947 seconds
*though this one was skipped
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file.
To test:
1. Download the joined MARC file containing 5 records.
2. Run the import command:
./misc/migration_tools/bulkmarcimport.pl -b --file=marc_file.mrc
3. Check the final script report
==> 11 MARC records done in 0.566349983215332 seconds
Expected output should show 5 records
4. Apply the patch
5. Repeat step 2, 3
==> The correct number is shown
Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Add Koha::Script -cron to restrict_patrons_with_failed_notices.pl
and sftp_file.pl which used cronlogaction directly without importing
Koha::Script. Remove their now-redundant manual calls.
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Remove start/end cronlogaction calls and unused C4::Log imports
from scripts that use Koha::Script, as these are now handled
automatically by the INIT/END blocks.
Scripts with mid-execution informational logging (plugins_nightly,
harvest_oai, fines, process_message_queue) retain their C4::Log
import and non-start/end cronlogaction calls.
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Add INIT and END blocks to Koha::Script that automatically call
cronlogaction on script start (with command line options) and
completion. This removes the need for scripts to manually handle
cron log entries.
This will mean that any script importing Koha::Script will be logged,
but I think the addition is beneficial.
To test:
1 - Ensure CronjobLog is enabled
2 - perl misc/cronjobs/fines.pl --test
3 - Confirm start and end are logged in action logs:
http://localhost:8081/cgi-bin/koha/tools/viewlog.pl?src=&user=&modules=CRONJOBS&actions=&object=&info=&interfaces=&from=&to=&output=screen&basename=Export&report_name=&do_it=1
4 - Apply patch
5 - Repeat 2
6 - Reload the logs, confirm the new logs look the same
7 - Sign off
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Always include the plural version, not the singular one (the plural
imports the singular)
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch replaces all calls to C4::Stats::UpdateStats with the wrapped
function from the Koha namespace. As this is the last function in C4::Stats, the
module is also deleted.
Several tests depended on transitive includes to Koha::Statistics through
C4::Stats. I've identified uses of that module by searching, and added the
necessary use in each case. I've run `prove -r t xt` with this patch applied and
didn't hit any relevant errors.
Test plan:
1. Inspect the patch, and verify each changed call maintains the same parameters.
2. Apply patch.
3. Prove the following files:
- t/db_dependent/Accounts.t
- t/db_dependent/Circulation.t
- t/db_dependent/Koha/Account.t
- t/db_dependent/Koha/ILL/Request.t
- t/db_dependent/Koha/Item.t
- t/db_dependent/Koha/Pseudonymization.t
- t/db_dependent/Koha/Recalls.t
Sponsored-by: Open Fifth
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch adds two new parameters to control the deletion of background jobs that did not complete.
Some libraries may want to keep a failed job longer than a completed one for debugging reasons.
To test:
0 - Apply patch
1 - Make some old failed jobs:
INSERT INTO background_jobs (status,type,enqueued_on) VALUES
('started','update_elastic_index','2020-01-01'),
('failed','update_elastic_index','2026-01-01'),
('started','other_type','2021-01-01'),
('failed','other_type','2026-01-01');
2 - Confirm defaults of 365 and update_elastic_index for new params:
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days
3 - Should indicate 1 job older than 365 of type update_elastic_index would have been deleted
4 - Confirm days param works
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days 30
5 - Shoudl find 2 elastic jobs
6 - Confirm type param
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days --jobs-unfinished-type=other_type
7 - Should find 1
8 - Confirm type and days
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days 30 --jobs-unfinished-type=other_type
9 - Should find 2
10 - Confirm repeated type
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days=30 --jobs-unfinished-type=other_type --jobs-unfinished-type=update_elastic_index
11 - Finds 4
12 - Confirm deletion
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days=30 --jobs-unfinished-type=other_type --jobs-unfinished-type=update_elastic_index --confirm
13 - All your made up jobs are removed
Signed-off-by: Trevor Diamond <trevor.diamond@mainlib.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Two follow-up fixes from QA review of the previous commit:
- Detect Search::Elasticsearch::Error::NoNodes via ref($_) rather than
regex-matching the stringified exception, matching the existing
pattern already used in about.pl.
- Only reset jobs to 'new' for retry when NoNodes is the sole problem.
If a genuine indexing error also occurred in the same batch, record
the batch as 'failed' instead of silently discarding that error via
the NoNodes retry path.
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When update_index() throws, the daemon now:
- On NoNodes: aborts the batch, recreates the indexer object to reset
the dead-node connection pool, and resets jobs to 'new' for retry.
- On other errors: marks jobs as 'failed' instead of 'finished'.
This prevents silent data loss where records disappear from search
results because their indexing jobs were incorrectly marked as done.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Note: this requires a command line access to the test machine
TEST PLAN:
1 - Anywhere on the computer running koha, run following command:
mkdir tmp
echo '<!DOCTYPE html><html><head><meta charset="UTF-8"></head><body><p>Тетрис — это классика!</p></body></html>' > tmp/tmp.html
echo '* { color: red };' > tmp.css
./misc/cronjobs/printoverdues.sh tmp tmp.css
2 - Open the pdf in tmp/tmp.pdf, notice it is broken
3 - Apply patch
4 - Rerun ./misc/cronjobs/printoverdues.sh tmp tmp.css
5 - Open the pdf in tmp/tmp.pdf, notice it is now displaying properly, notice the css property still applies
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Bug 40866 introduced JSON logging for HOLDS CREATE but stored null for
'confirmations' and 'forced' when no overrides were present. This patch
mirrors the earlier change to CIRCULATION ISSUE by defaulting them to
empty arrays, keeping the two log streams structurally identical and
simplifying reporting queries.
Changes:
C4/Reserves.pm:
- Default confirmations/forced to [] in the AddReserve log payload.
t/db_dependent/Reserves.t:
- Update the Bug 40866 subtest to assert the empty-array defaults on
normal holds, and fix a 'my $log_data' redeclaration.
tools/viewlog.pl:
- Extend the CSV export JSON-decode block to also cover HOLDS logs,
guarding the item lookup on itemnumber being present (HOLDS payloads
may lack one) and filling the CSV item columns from the decoded
payload. The legacy bare-itemnumber fallback is restricted to
CIRCULATION so a legacy HOLDS hold_id is not mistaken for an
itemnumber.
koha-tmpl/intranet-tmpl/prog/js/viewlog.js:
- Extend the info column override rendering (renderCircInfo) to HOLDS
entries so forced/confirmations arrays display translated, exactly as
they do for CIRCULATION. The log viewer moved to an ajax datatable in
bug 29016, so this replaces the earlier json_found change to
viewlog.pl/viewlog.tt.
misc/maintenance/migrate_action_logs_to_json.pl:
- Extended to handle legacy pre-Bug 40866 HOLDS CREATE logs where info
is a bare hold_id, recovering branchcode / biblionumber / itemnumber
from old_reserves / reserves.
- Uses the same keyset-pagination pattern as the circulation loop so
rows are not skipped under --commit.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch adds an out-of-band migration script to convert legacy
CIRCULATION ISSUE action logs from the old itemnumber-only format
to the new consistent JSON format.
The script:
- Finds all CIRCULATION ISSUE logs where info is just an itemnumber
- Converts them to JSON format with the standard structure
- Attempts to recover issue_id and branchcode from old_issues/issues
- Runs in dry-run mode by default (--commit flag to actually update)
- Processes in batches to handle large databases safely
- Includes verbose mode for detailed output
Usage:
perl misc/maintenance/migrate_action_logs_to_json.pl --help
perl misc/maintenance/migrate_action_logs_to_json.pl --verbose
perl misc/maintenance/migrate_action_logs_to_json.pl --commit
This is designed as an out-of-band maintenance script rather than
an atomic update because:
1. May be expensive on large databases with many log entries
2. Migration is optional - display code handles both formats
3. Can be run during off-peak hours at admin's discretion
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Like Bug 32012 for runreport.pl the script search_for_data_inconsistencies.pl should use binmode UTF-8 to correctly display datas.
Test plan:
1) Edit an authorised value to add some diacritics in code, in "LOC" for example
2) Edit an item to use this code
3) Delete this authorised value
4) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Check you have a correct message with diacritics
Signed-off-by: Chris Mathevet <chris.mathevet@inLibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
To test:
Test the cronjob purge_suggestions.pl
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The stockrotation cronjob was only sending email reports when there were
actionable items (items that needed to be moved between branches). This
meant that libraries would not receive any notification when all items
in their rotation were stationary, even when the --send-all flag was
passed to the script.
This patch modifies the report logic to also send reports when there are
only stationary items, ensuring libraries have visibility into the complete
status of their stock rotation items.
Test plan:
1. Set up a stock rotation rota with items that are all stationary
(items that should stay at their current branch)
2. Run the stockrotation cronjob: perl misc/cronjobs/stockrotation.pl
3. Without this patch: No email report is generated
4. Apply this patch
5. Run the cronjob again
6. Verify that an email report is now generated showing the stationary items
7. Test with mixed scenarios (some actionable, some stationary items)
8. Confirm reports are sent in all cases where items exist in the rotation
Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Correct genuine spelling errors identified by the codespell 2.4.x
full scan after the KTD container OS upgrade. Changes are in
comments, POD, strings, and template text across 55 files.
"SAN Ouest Provence" (the French library co-operative and Koha
sponsor) is preserved as a proper noun. Perl copyright lines
receive an inline # codespell:ignore annotation; about.tt files
(contributor attribution pages) are added to the Koha::Devel::Files
codespell exception list so that future uses of "Provence" as a
typo for "Province" are still caught in all other files.
Notable user-visible fixes:
- "submiting" -> "submitting" in authority and bibliographic merge screens
- "deleteable" -> "deletable" in patron exception messages
- "Acquistions" -> "Acquisitions" in reports dictionary
Test plan:
1. Apply patches
2. In the staff interface, go to Cataloguing > Merge records and
confirm the submit button reads "Submitting" (not "Submiting")
3. In the staff interface, go to Authorities > Merge and confirm
the same correction
4. Attempt to delete a patron with outstanding issues and confirm
the error message reads "deletable" (not "deleteable")
5. prove xt/author/codespell.t
=> All tests pass
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This is part of a sponsored piece of development from one of our customers. There was a specification of requirements but I was conscious of the ideas already discussed on this bug and wanted to try and develop a solution that would balance both sets of requirements. The concept works as follows:
- Self-renewal can be enabled and controlled at a category level
- The window within which it is available is customisable (both pre and post-expiry)
- Defaults to the NotifyBorrowerDeparture syspref
- The amount in fines that block self-renewal is customisable
- Defaults to the category level charges and then the noissescharge preference
- The message to display if renewal fails is customisable
- Libraries can also add information to display to the patron at the start of the process.
- The self-renewal process will also respect the OPACPatronDetails and AutoApprovePatronProfileSettings sysprefs. If OPACPatronDetails is enabled, the renewal will redirect the patron to the OPAC details form and ask them to confirm their details are still correct. Any changes will be logged as patron modifications and approved in line with the auto approval syspref
- Renewal happens regardless of whether OPACPatronDetails is enabled, only the details changes are logged as modification requests
- Notifications are sent to the patron in line with their messaging preferences
Test plan:
1. Apply all patches and reset_all
2. In the intranet, navigate to patron categories and edit the ‘Patron’ category
3. Scroll to the bottom, there should be a section called ‘Account expiry and self-renewal’
4. Set ‘Allow self-renewal’ to Yes
5. Set Self-renewal availability period to be 10 days
6. Set Self-renewal availability post-expiry’ to also be 10 days and save
7. In system preferences, set the OPACPatronDetails preference to ‘Don’t allow’
8. Choose a patron that has the ‘Patron’ type category
9. Edit that patron so that you can log into the OPAC and set their expiry date to be yesterday’s date
10. Log into the OPAC as that patron (it may be easiest to do this in incognito as we will be swapping between the intranet and OPAC throughout the test plan and it will remove the need to keep logging out and in)
11. You should see a message that says “You are eligible for self-renewal. Please click here to renew your account.”
12. Click that link - you will be presented with a modal that says “Are you sure you want to renew your account?”
13. Click “Close” - we don’t want to complete renewal yet as there is more functionality available to test
14. Navigate to the patron category again, set ‘Self-renewal availability post-expiry’ to be 0
15. Refresh the OPAC, the self-renewal link should not be there as we are now outside the patron self-renewal window
16. Undo that change in the patron category, the link should reappear when refreshing the OPAC
17. Now set the patron’s expiry date to be tomorrow
18. In the category, set ‘Self-renewal availability period’ to be blank
19. Refresh the OPAC, it should still show the link as it is defaulting to NotifyBorrowerDeparture
20. Set NotifyBorrowerDeparture to 0
21. Refresh the OPAC, the link should vanish
22. In the category, restore ‘Self-renewal availability period’ to 10 days
23. Refresh the OPAC, the link should reappear
24. In the intranet, set the patron to be debarred using a manual restriction
25. In the OPAC refresh the page - self-renewal should no longer be available
26. Remove the restriction - the renewal link will be visible again
27. Set a fine on the patron record of 6 USD
28. Refresh the OPAC, the self-renewal link will be gone because we are over the global ‘noissuescharge’ preference value of 5
29. In the category, set ‘Checkout charge limit’ to 7
30. Refresh the OPAC, the link will be visible because we are now under the category charge
31. Add an additional fine of 2 to the patron account
32. Refresh the OPAC, the self-renewal link is gone
33. In the category, set ‘Self-renewal charge limit’ to be 10
34. Refresh the OPAC, the link will be visible again
35. In the category, set ‘Self-renewal failure message’ to be whatever you want
36. In the OPAC, refresh the page and then click into the self-renewal process
37. This time, click “No” - you should see the failure message you have just set
38. Click “Close” and refresh the page to reset this message
39. In the category, set a message in the “Self-renewal information message”
40. In the OPAC, refresh the page and then click the renewal link again
41. You should be met with the modal, this time displaying the information message we just set up in the patron attributes
42. Click “Yes” - you should see the “Are you sure you want to renew your account?” message again
43. This time when prompted with the “Are you sure” message, click “Yes”
44. The page should refresh and display a confirmation with the new expiry date.
45. In the intranet, view the patron’s record - the date should have been updated
46. In the Modification log tab, there should be a log entry for “Membership renewed”
47. Change the patron’s expiry date again, this time to be yesterday’s date
48. In the OPAC, click on “Personal detail”, the new expiry date should show there
49. Now set the OPACPatronDetails syspref to be “Allow”
50. Click on “Summary” again, the self-renewal link should be there again
51. Click into the self-renewal process and click “Yes” on your verification step
52. This time you should be met with a message that says “Confirm your account details”
53. Click “Continue” - you will be redirected to the OPAC details form
54. There should be a message at the top that says “Please verify your details to proceed with your self-renewal.”
55. Update a field somewhere on the patron form
56. Scroll to the bottom - the button should say “Submit renewal request”
57. Click this button
58. You will be shown the renewal confirmation message
59. In the intranet, on the homepage you should now see a modification request
60. Check the request, it should match the fields you changed - don’t approve or deny for now
61. Set ‘AutoApprovePatronProfileSettings’ in sysprefs to “Enable”
62. Reset your patron’s expiry date to today’s date
63. In the OPAC, click into Personal details and then back into Summary again (we need to do this to remove the success parameter from the URL)
64. The renewal link will be visible again - go through the process again
65. On the form, you should see an alert that you have already sent a modification request
66. This time, change a different field to the previous attempt
67. Submit the form
68. In the intranet there should be no modification this time and the change should be visible in the patron record as it has been automatically approved and handled as per the syspref
69. prove t/db_dependent/Koha/Patron.t
70. prove t/db_dependent/api/v1/patrons_self_renewal.t
71. yarn cypress run --spec t/cypress/integration/Islands/SelfRenewal.ts
72. Sign off!
Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This looks trivial at first, but it's not, because of forking and the
comment in C4::Context
I don't think the comment makes sense, we need to retrieve a new dbh,
and so passing the 'new' parameters to C4::Context->dbh should do the
same.
Test plan:
Try rebuild_zebra.pl and confirm there is no error
% perl misc/migration_tools/rebuild_zebra.pl --processes 2 -b
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds a new option to define the providers in cronjob erm_run_harvester.pl
that are used to run the SUSHI harvesting for COUNTER Reports in the ERM module.
Use parameter --provider-id or -p
The parameter provider_id is repeatable.
If the parameter for provider_id is not used all active providers will be harvested (as before).
The script will check if these providers are active (as before).
Script to run on ktd: perl misc/cronjobs/erm_run_harvester.pl --begin-date 2025-09-01 --end-date 2025-09-30
To test:
- Run the script and make sure that all active providers are harvested.
- Apply patch.
- Run the script with only one provider-id.
- Run the script with two provider-ids by using -p 1 -p 2
- Run the script without the parameter.
- Make sure that in all cases only active providers are harvested.
- Sign off :-).
Sponsored-by: Karlsruhe Institute of Technology (KIT)
Signed-off-by: Jan Kissig <bibliothek@th-wildau.de>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
- Add missing PrintSlipCSS accordion panel to the notice editor
- Pass AllNoticeCSS and PrintNoticeCSS to gather_print_notices.pl HTML output
- Include inline <style> blocks in print-notices.tt template
Signed-off-by: Marion Durand <marion.durand@biblibre.com>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan (for --cards arguments) :
1. Initial Setup:
a) Go to Tools > Patron card creator > New > Card batch
b) Create two different batches (Batch 1 and Batch 2) using the 'Add by borrowernumber(s):' option.
If using **KTD** you can use the following borrowernumbers for both batches:
15
29
47
else you will need to create your own patrons and enter their borrowernumber.
c) In your terminal, set the timestamps manually:
--OPTION A (KTD users):
ktd --shell
koha-mysql kohadev
--OPTION B:
Access your database
Then:
UPDATE creator_batches SET timestamp = DATE_SUB(NOW(), INTERVAL 30 DAY) WHERE batch_id = '1';
UPDATE creator_batches SET timestamp = DATE_SUB(NOW(), INTERVAL 5 DAY) WHERE batch_id = '2';
Note: your batch_id may differ.
2. Before Applying the Patch:
a) Run the following cleanup script:
./misc/cronjobs/cleanup_database.pl --cards 15 --confirm --verbose
b) Error in console output
c) Nothing is deleted even though one batch was older than 15 days
SELECT * FROM creator_batches;
3. Apply The Patch
4. After Applying The Patch:
a) Redo step 1 (Initial Setup).
b) Run again the following cleanup script:
./misc/cronjobs/cleanup_database.pl --cards 15 --confirm --verbose
c) Check that the console output says: "Purging ... more than 15 days ago"
d) Batch 1 is deleted, but Batch 2 (5 days old) is STILL PRESENT. Success!
SELECT * FROM creator_batches;
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan :
1. Initial Setup:
a) Go to Cataloging > Label creator > New > Label batch
b) Create two different batches (Batch 1 and Batch 2) using the 'Add by barcodes or itemnumbers' option.
If using **KTD** you can use those barcodes for the two batches:
39999000010114
39999000010138
39999000010152
else you will need to create your own records with items and enter their barcodes.
c) In your terminal (in a ktd --shell), set the timestamps manually:
--OPTION A (KTD users):
ktd --shell
koha-mysql kohadev
--OPTION B:
Access your database
Then:
UPDATE creator_batches SET timestamp = DATE_SUB(NOW(), INTERVAL 30 DAY) WHERE batch_id = '1';
UPDATE creator_batches SET timestamp = DATE_SUB(NOW(), INTERVAL 5 DAY) WHERE batch_id = '2';
Note: your batch_id may differ.
2. Before Applying the Patch:
a) Run the following cleanup script:
./misc/cronjobs/cleanup_database.pl --labels 15 --confirm --verbose
b) Check that the console output says: "Purging ... more than 1 days ago"
c) Check database and notice that it is now empty, even though one set was less than 15 days old.
SELECT * FROM creator_batches;
3. Apply The Patch
4. After Applying The Patch:
a) Redo step 1 (Initial Setup).
b) Run again the following cleanup script:
./misc/cronjobs/cleanup_database.pl --labels 15 --confirm --verbose
c) Check that the console output says: "Purging ... more than 15 days ago"
d) Batch 1 is deleted, but Batch 2 (5 days old) is STILL PRESENT. Success!
SELECT * FROM creator_batches;
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
HTML1 is gone. Almost...
The previous rule said that we could not have TT tags within HTML tags
because it caused problems with the translation.
It was incorrect.
Bug 38720 removed that rule, and stated that the only problematic use
was a IF with ">".
It was also incorrect.
This one bug states that we cannot have TT tags to construct HTML tags
IF one of the attribute contains a translatable string.
The list of translatable attributes are listed in tmpl_process3.pl.
This patch restores xt/tt_valid.t (and move it to xt/author) and adjust
slightly the test to take into account this last statement.
Test plan:
0. Apply only this first patch
1. Run xt/author/tt_valid.t
=> It fails on mana-subscription-search-result.inc
2. Update po for your prefered language
=> The string "This resource has been reported more than" is not in the
po files
3. Apply the second patch, repeat 1 and 2
=> It still fails: adding a space between the TT tag and the attribute is not enough here
=> The string is picked now! but... install the template and open the
translated version =>
4. Apply the third patch, repeat 1 and 2
=> The tests pass!
=> The string is still picked, install => The template looks good!
Now if you are familiar with Mana you can try to see the translated
string on the UI
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
xgettext.pl tries to produce output to STDOUT by default, but fails
because of an error in the code. It should go to the actual STDOUT
instead of a file named '&STDOUT'.
Test plan:
1. Run xgettext.pl:
echo koha-tmpl/opac-tmpl/bootstrap/en/includes/serial-status.inc > transfiles
./misc/translator/xgettext.pl -vaf transfiles
2. Observe no standard output, and the existence of a file
called `&STDOUT`. Output to the screen is:
echo koha-tmpl/opac-tmpl/bootstrap/en/includes/serial-status.inc > transfiles
kohadev-koha@kohadevbox:koha(main)$ ./misc/translator/xgettext.pl -vaf transfiles
./misc/translator/xgettext.pl: Outputting to STDOUT...
./misc/translator/xgettext.pl: Opening input file list "transfiles"
./misc/translator/xgettext.pl: Processing file "./koha-tmpl/opac-tmpl/bootstrap/en/includes/serial-status.inc"
3. Apply the patch.
4. Run again.
5. Observe that information is output to the screen (see example of
outpunt in the bug comment).
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Standardizes error code spelling across the codebase:
- Changes 'too_much_oweing' to 'too_much_owing'
- Changes 'auto_too_much_oweing' to 'auto_too_much_owing'
The word "owing" is the correct spelling in English.
Files updated:
- C4/Circulation.pm
- C4/SIP/Sip/MsgType.pm
- circ/renew.pl
- koha-tmpl/intranet-tmpl/prog/en/modules/circ/renew.tt
- koha-tmpl/intranet-tmpl/prog/js/checkouts.js
- koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-reserve.tt
- koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-user.tt
- misc/cronjobs/automatic_renewals.pl
- opac/opac-reserve.pl
- opac/opac-user.pl
- t/db_dependent/Circulation.t
All tests should pass successfully.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
If PatronSelfRegistration we should ignore (i.e. do nothing) if
--del-exp-selfreg is passed to cleanup_database.pl
It prevents the script to warn that the pref is not set.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Martin Renvoize
Kyle M Hall
Nick Clemens
Tomás Cohen Arazi
Chloé Zermatten
Martin Renvoize
Katrin Fischer
John Doe
Lari Strand
Jan Kissig
Hammat Wele
Jonathan Druart
Olivia Reynolds
Baptiste Wojtkowski
Fridolin Somers
Lucas Gass
Matt Blenkinsop
Raphael Straub
Emily Lamancusa
Ayoub Glizi-Vicioso
Victor Grousset/tuxayo
Tadeusz „tadzik” Sośnierz