The angle-bracket delimiter notation in the POD prose (e.g.
PATTERN<DELIM>) caused the POD parser to treat the trailing letter of
each word as an unknown formatting code (N<>, T<>). Replace with
C<s/PATTERN/REPLACEMENT/FLAGS> and a parenthetical note that any
non-space delimiter is accepted.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
When eval { qr/$pattern/ } fails (e.g. the admin-configured s/// contains
an invalid regex), _apply_substitution previously returned silently with no
indication of why the rule was skipped. Add a Koha::Logger warn so the
failure appears in the log and is diagnosable without a debugger.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Note: While the methodology in Koha::Regex::Replacement may seem
bespoke, it's similar to how Python's re.sub(pattern, repl, string)
regex substitution works.
Unit tests:
prove -v t/Koha/Regex/Replacement.t \
t/SimpleMARC.t \
t/db_dependent/MarcModificationTemplates.t \
t/Labels_split_Regex.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
C4::ClassSplitRoutine::RegEx::split_callnumber ran the admin-configured rule
through eval "\$cn_item =~ $regex", executing arbitrary Perl. Parse the
rule as s<DELIM>PATTERN<DELIM>REPLACEMENT<DELIM>FLAGS, reject the /e flag,
compile the pattern with qr// and expand the replacement as data via
Koha::Regex::Replacement::expand_template, so the rule is applied without
eval.
Signed-off-by: Sanjar Tulkinov Anvar o'g'li <sanjarbiy11@gmail.com>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
The original fix only whitelisted agefield, condition_field, and
substitution_field at the point they're saved in
tools/automatic_item_modification_by_age.pl, leaving the actual
consumer of the stored syspref, C4::Items::ToggleNewStatus, trusting
agefield and substitution field unchecked. agefield is spliced
directly into raw SQL (TO_DAYS($agefield)) and so can't be
parameterized like an ordinary value; substitution field reaches a
dynamic method call ($item->$field($value)). Any future or
alternate writer of the automatic_item_modification_by_age_configuration
syspref would have reopened both holes without touching the form at
all.
This adds the same allow-list validation directly in ToggleNewStatus
(falling back to the safe default agefield, or skipping the
substitution/condition, exactly as before for missing values), shares
the age-field allow-list between the two files via a new
GetAutomaticItemModificationAgeFields() in C4::Items so it can't drift
between them, and logs rejected values via Koha::Logger instead of
silently discarding them so tampering leaves an audit trail.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
login_core only cleared leftover session state on the failed-auth
branch. The earlier 'SIP access not authorized' return (unknown
terminal id) left any account/ils/institution/policy/credentials from
a prior connection on a reused prefork worker in place.
Move the reset to the top of login_core so every non-success exit path
leaves clean state, and only a successful api_auth re-establishes the
session. This keeps the no-authenticated-state-until-auth invariant in
a single place rather than duplicating the key list per branch.
Also extend the unit test to cover the unknown-terminal path on a
reused worker.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
After a SIP terminal logs in successfully, a later connection handled by the
same worker could bypass authentication entirely. login_core() stores the account
in the session before calling api_auth(), and left it in place when authentication
failed. The check in raw_transport() that is meant to drop unauthenticated connections
only looks at $self->{account}->{id}, which is now populated straight from the SIP config,
so the connection is treated as logged in.
Test Plan:
1) Create a SIP user with the username and password "term1"
2) Restart the SIP server
3) Apply the first patch
4) prove t/db_dependent/SIP/MsgType.t
5) Note the new test fails
6) Telnet to the SIP server port ( 6001 for ktd )
5) Send a successful login: 9300CNterm1|COterm1
6) Note you get a 941 back
7) Disconnect, then telnet to localhost 6001 again
8) Send a bad login: 9300CNterm1|COterm2
9) Note you get a 940 back, but the connection stays open
10) Send a patron information request, using a real patron cardnumber:
6300120260615 110310 AOCPL|AA23529001000463
11) Note you get a 64 patron information response with the patron's data back,
even though the login failed!
12) Apply the second patch
13) prove t/db_dependent/SIP/MsgType.t
14) Note the subtest now passes!
15) Restart all the things!
16) Repeat steps 4 to 10
17) Note the the patron info request returns no patron data and the connection is dropped!
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
In Koha, the sip2_resource_last_modified key is updated any time the SIP config is edited from the SIP2 editor in the staff interface ( institutions, accounts, system preference overrides ).
If sip2_resource_last_modified is missing from memcached, Koha SIP emits "Couldn't find sip2_resource_last_modified, considering configuration not up to date" in the logs, and fetches the entire configuration from the database, *for each request*.
This causes SIP performance to drop significantly. In my production testing with 83 institutions and 171 accounts the login message response time dropped from 0.283 seconds to 2.899 seconds.
I think the solution is pretty simple. If there is a cache miss on sip2_resource_last_modified, SIP should load the config from database as it does already, but then it should also set sip2_resource_last_modified!
Test Plan:
1) Apply this patch
2) Prove t/db_dependent/Koha/SIP2/SIP2ModuleMigration.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 7dc223b30a)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Make 'Maximum total holds allowed (count)' (aka max_holds) consider hold groups
Test plan:
Same as the previous test plan, but set 'Maximum total holds allowed (count)' as 2 and instead set 'Holds allowed (total)' as unlimited.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e56c9cc9bb)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Make 'Holds allowed (total)' (aka reservesallowed) consider each hold group as an individual hold
Test plan:
1) Enable DisplayAddHoldGroups sys pref
2) Set Holds allowed (total) to 2, visit:
http://localhost:8081/cgi-bin/koha/admin/smart-rules.pl
3) Scroll to the right, click the 'Edit' button under 'Actions' column.
4) On the 'Holds allowed (total)' column cell, enter '2' and click 'Save' under the 'Actions' column to the right.
5) Search for 'test':
http://localhost:8081/cgi-bin/koha/catalogue/search.pl?q=test
6) Tick the checkbox next the first and second results, click 'Place hold' button at the top toolbar
7) Search for 'koha' user and pick Koha (42)
8) Set a pickup location for each hold and tick the 'Treat as hold group' checkbox. Click 'Place holds'.
9) Try to place another single hold for the same patron. Should succeed, because the 2 grouped holds count as 1. Patron now should have 1 group + 1 ungrouped hold = effective count of 2.
10) Try to place yet another hold, should be blocked with a tooManyHolds error.
11) Prior to this patch, step 9 would've failed because the 2 grouped holds counted as 2.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit d5f130da5a)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
The "Cite" option in the OPAC is missing the subtitle (245) in MARC21.
To recreate:
1. Add a record or find a record with a subtitle (in KTD, biblionumber 58 has a 245 http://localhost:8081/cgi-bin/koha/catalogue/MARCdetail.pl?biblionumber=58
2. In the OPAC, go to the record
http://localhost:8080/cgi-bin/koha/opac-detail.pl?biblionumber=58
3. Click "Cite"
--> The subtitle is not in any of the citations
4. Apply the patch
5. Repeat step 2 and 3
--> The subtitle is now shown
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 412e4e4d95)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Test plan:
Read the patch.
Search for both parameters to verify that they are not used.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit f5a38efff1)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit db6287aa21)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
UNIMARC instances include the following two authority types
by default (among others): Corporate Body Name (code: 'CO')
and Collective Title (code: 'CO_UNI_TI').
The problem is that when searching for Corporate Body Name
authorities (in the OPAC or in the Staff interface), the
results will include Collective Title authorities as well.
It has to do with Corporate Body Name authorities using
an authtypecode (CO) that happens to be the prefix of the
authtypecode (CO_UNI_TI) used by Collective Titles.
NOTE: Elasticsearch is not affected by this.
The fix is to additionally pass the completeness attribute
to the RPN query sent to Zebra so that it will try to match
on what is literally found in the entire field's phrase
index (i.e. the value of UNIMARC 152$b), instead of performing
a partial match.
More information can be found at the official documentation:
https://software.indexdata.com/zebra/doc/querymodel-rpn.html#querymodel-bib1-completeness
Test plan (tailored for KTD):
0) Launch a UNIMARC Koha instance using Zebra:
ktd --marcflavour unimarc --search-engine zebra up
1) Import the Sample MARC record attached to this Bug,
containing a single Collective Title authority
(adding one manually would take a long time due to
the many mandatory subfields).
2) Perform a generic authority search (without entering
anything in the search box) for all 'Corporate Body
Name' authorities. Note the number of results returned
(83), and also notice that in page 4 of the results you
get a stray 'Collective Title' authority ("Selected
works", the one you imported in Step 1).
3) Apply this patch (and `restart_all`).
4) Repeat the same search as in Step 2. This time, the
number of results returned should be one less than
before (82), and you should only see 'Corporate Body
Name' authorities in the results.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e43d32a123)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
This patch updates the diff for ADD/CREATE and DELETE logactions to
exclude undefined fields from the diff.
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit d095599447)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Update suggestion action logging to populate the diff column using the
correct before/after semantics:
- CREATE: empty before-state ({}), new suggestion data as after-state
- MODIFY: pre-change unblessed data as before, post-store object as after
- DELETE: log before the SQL DELETE so the object can still be fetched;
pass the suggestion as both $infos (info column) and $original (diff
before-state), logaction sets after-state to {}
Also add tests verifying the diff column is populated for all three actions.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit efe6f26e41)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
This commit improves upon the original fix by making the intent more
explicit when handling empty string date parameters in AddRenewal and
AddIssue functions.
WHY THIS IS BETTER:
The original fix (changing `defined $datedue` to just `$datedue`)
works correctly but relies on Perl's implicit truthy/falsy semantics:
if ( $datedue && ref $datedue ne 'DateTime' )
This is subtle because:
- Empty string "" is falsy, so parsing is skipped ✓
- But it's not immediately obvious that empty strings are being
handled specially
- The code doesn't explicitly show its intent
THIS APPROACH:
if ( defined $datedue && $datedue ne '' && ref $datedue ne 'DateTime' )
Benefits:
1. EXPLICIT: Clearly states "only parse if defined AND not empty
AND not already a DateTime"
2. SELF-DOCUMENTING: Intent is obvious to future developers
3. NO RELIANCE on truthy/falsy semantics
4. CONSISTENT: Same pattern applied across all date parameter handling
COMPREHENSIVE CHANGES:
1. AddRenewal (C4/Circulation.pm:3449)
- Made datedue empty string check explicit
2. AddIssue (C4/Circulation.pm:1611)
- Made datedue empty string check explicit
- Also updated issuedate handling (line 1616) to explicitly handle
empty strings
3. Test Coverage (t/db_dependent/Circulation/issue.t)
- Added test for AddIssue with empty string datedue
- Added test for AddIssue with empty string issuedate
- Ensures both functions properly calculate dates from circulation
rules instead of using "now"
The existing test for AddRenewal already covered empty string datedue
(added in the original Bug 37966 commit). These new tests ensure the
same correct behavior for AddIssue.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 14de13d3ad)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
When renewing a book on hold with the RenewalOnHoldOverride sys pref set
to "Allow", if the due date is not specified then it becomes "now". By
checking the input is set rather than just defined the due date is set as
expected based on circulation rules.
Also change the description of the RenewalOnHoldOverride sys pref as the
due date would no longer be required to be entered with this change.
Test plan:
1. make sure RenewalOnHoldOverride is set to 'Allow'
2. check out a book, then place a title-level hold on it
3. renew the book, see the hold override dialog
4. leave the due date empty and hit the override button
5. observe the new due date is "now"
6. apply patch and restart plack
7. renew the book again, overriding the hold, and see the expected due date
as per circulation rules.
Signed-off-by: Marion Durand <marion.durand@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit c84de245e3)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
The previous POD follow-up added headings for the private helper subs but
introduced several typos and used =head2, which puts them at the same level
as the public API in the generated docs. This patch:
- Fixes typos: matchpint, pionts, march matcher, mathcpoints, etc.
- Clarifies a couple of descriptions.
- Demotes each private helper from =head2 to =head3 so they render as
implementation notes rather than peers of the public methods, matching
the pattern used in Koha::Authority's "Internal methods" block.
Helper subs remain interleaved with the public API in source order to keep
the diff minimal; a wholesale move into a dedicated "Internal functions"
section would be a larger refactor outside the scope of this bug.
Test plan:
1 - perldoc C4/Matcher.pm
2 - Confirm no typos are rendered and the private helpers appear one level
deeper than the public methods.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 9f1ae44cc7)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
The previous defensive block still dereferenced $target_auth unconditionally
when attempting to recover via record_strip_nonxml. If Koha::Authorities->find
returned undef (stale search index pointing at a deleted authid) the eval
around ->record would catch the first failure, but the recovery call on the
next line would die uncaught.
This patch:
- Skips silently when the authid returned by the search engine no longer
exists in the database, so get_matches cannot die on a stale index.
- Wraps record_strip_nonxml in eval so an exception there is also handled
and the record is simply skipped - matching the stated intent of the
"skip when not possible" path.
Test plan:
1 - Run t/db_dependent/Matcher.t - still passes.
2 - Simulate a stale index: mock search_auth_compat to return an authid
that is not present in auth_header. Confirm get_matches no longer dies
and returns without error.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 9d2b2c2493)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 94a9712745)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
When matching authorities we can die on a bad record, we should add protection against this
on top of bug 41843
To test:
1 - Apply index patch and rebuild elastic
perl misc/search_tools/rebuild_elasticsearch.pl -r -v
2 - Make a new record matching rule
threshold: 1000
type: authority
index: authid
score: 1000
tag: 001
3 - Choose an authority record and corrupt it:
sudo koha-mysql kohadev
UPDATE auth_header SET marcxml = UpdateXML(marcxml, '//datafield[@tag="040"]/subfield[@code="c"]',CONCAT('<subfield code',CHAR(27),'="c">OSt</subfield>')) WHERE auth_header.authid=92;
4 - View and save the record as marcxml
http://localhost:8081/cgi-bin/koha/authorities/detail.pl?authid=92
5 - Cataloging - stage marc records - authority/marcxml/no matching
6 - Confirm it stages
7 - Attempt to match using your new rule
8 - Get an error parsing the marc
9 - Apply patch, restart all
10 - Relaod the staged batch
11 - Attempt matching agaion
12 - Success!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit c5adbe0818)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
The EAN-13 barcode query used max(abs(barcode)) which allowed non-EAN-13
barcodes (e.g. 14-digit KTD sample barcodes) to dominate the result.
The saved 13-digit EAN-13 barcodes were always smaller, so $nextnum
never advanced and the same barcode was generated on every page load.
Fix by extracting EAN-13 logic into C4::Barcodes::ValueBuilder::EAN13
(consistent with other barcode type packages) and scoping the query to
WHERE length(barcode) = 13 so only valid-length barcodes contribute to
the max. Both barcode.pl and Koha::REST::V1::Biblios now call the shared
function, eliminating the duplicated inline code.
Tests: adds EAN-13 test cases to t/db_dependent/Barcodes_ValueBuilder.t
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit fc6e5839d4)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Reports may consume too much resources of the db. To avoid that a new config param
'report_sql_max_statement_time_seconds' was created, which sets the maximum time a report statement may
take to execute in seconds. If it takes longer it will be aborted.
reflowed the code, so that is is executed after is_sql_valid, and put
the code in a subroutine
Test plan:
0. if you run this test in ktd you should check if you are running
mysql or mariadb, and then be able to switch database
1. create a new report by going to
More > Reports > New SQL Report
(/cgi-bin/koha/reports/guided_reports.pl?op=add_form_sql)
enter the following SQL (which only purpose is to run long)
select ExtractValue(bm1.metadata,'//controlfield[@tag="001"]')
from biblio_metadata bm1
join biblio_metadata bm2 on
ExtractValue(bm1.metadata,'//controlfield[@tag="001"]') =
ExtractValue(bm2.metadata,'//datafield[@tag="773"]/subfield[@code="w"]')
give the Report a name eg "extract_long"
2. run the report
3. report should be executed and return several 1000 lines on ktd
4. apply patch
5. add a the parameter to koha-conf.xml to the <config> </config> part
<report_sql_max_statement_time_seconds>0.001</report_sql_max_statement_time_seconds>
OR use the following statemtn
sed -i '/<biblioserver>/a <report_sql_max_statement_time_seconds>0.001<\/report_sql_max_statement_time_seconds>' /etc/koha/sites/kohadev/koha-conf.xml
6. restart koha
koha-plack --restart kohadev
7. go back to the report page and rerun report which now should lead to an error
The following error was encountered:
The database returned the following error:
Query execution was interrupted, maximum statement execution time exceeded
Please check the log for further details.
8. in koha-conf.xml set
<report_sql_max_statement_time_seconds>20</report_sql_max_statement_time_seconds>
9. restart koha
10. rerun query, which now should execute as before
11. stop ktd, and restart with the "other" database
DB_IMAGE=mysql:8.0 ktd up
DB_IMAGE=mariadb:10.5 ktd up
12. repeat from step 1 (omit applying the patch)
13. Sign off
14. Thx
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 07ee120d50)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit c6ef3094ca)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
This silences warnings about undefinved values in the cache key
This seems to come from some calls in _build_languages_arrayref where we might not
have all variants of a language - the call could be prevented, but I am not sure
what the effects would be, so I simply convert undef (null value) to "undef" (string)
and use that as the cache key
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 365770c396)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
This should have no visible effect, just internal caching
To test:
1 - Do a search, check the language facets
2 - Apply patch, restart all
3 - Do a search, check the language facets
4 - No change
5 - Sign off
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 694765da2f)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 6393701c2a)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Koha's XSLT display path now calls a plugin hook so plugins can
extend the record processor filter list before MARC is transformed.
This enables prototype punctuation filters (and similar) to run
as plugins without core changes.
Test plan:
1) Enable plugins in koha-conf.xml.
2) Install the TestPunctuation plugin from https://github.com/HKS3/koha-test-punctuation
3) Enable the plugin in Koha admin.
4) View a biblio in staff or OPAC with XSLT display enabled and confirm
the filter runs (e.g., 245$a gets a trailing period if missing).
Signed-off-by: Alexander Wagner <alexander.wagner@desy.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit bb0d94b261)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
When merging (or updating) authorities, the biblios using the auths might also be updated. If this happens, Koha used to create one background job for each biblio. If an auth is used in a lot of biblios, this creates a lot of background jobs, each to index one single biblio.
This patch fixes that by collecting the affected biblios and putting all of them into one reindex background job.
Test plan:
* Apply the patch
* run the unit test t/db_dependent/Authority/Merge.t
* the test should now have 16 tests in total, with a new test #15 output like this:
1..3
ok 1 - 3 bibliographic record updated while merging two authorities
ok 2 - we have only one more background job
ok 3 - we have the correct biblio ids in the job to be indexed
ok 15 - merge with ModBiblio only creates one indexer background job for multiple biblios
ok 16 - no warnings
Manual test plan (rather cumbersome):
* Start KTD with ElasticSearch
* Create an PERSON authority (eg "100a Test")
* Create three biblios which use that authority in eg 100
* Access the DB to get the count of background jobs
* `ktd --dbshell`
* `select count(*) from background_jobs;`
* the count should be the same as the number of biblios you created (eg 3). But whatever it is, remeber it
* Edit the authority (change 100 to "Test new")
* check if the change is reflected in the linked biblios (100a = "Test new")
* Check the Background jobs in the DB again:
* `select count(*) from background_jobs;`
* should be three more than previously (eg 6)
* You can use this SQL to verify that the last three jobs each contain one biblionumber (which is what we want to fix):
* `select json_extract(data,'$.record_ids'),enqueued_on from background_jobs order by enqueued_on desc limit 3;`
+-----------------------------------+---------------------+
| json_extract(data,'$.record_ids') | enqueued_on |
+-----------------------------------+---------------------+
| [2299] | 2026-03-25 13:19:34 |
| [2300] | 2026-03-25 13:19:34 |
| [2301] | 2026-03-25 13:19:34 |
+-----------------------------------+---------------------+
* Now apply the patch and run restart_all
* Edit the authority again, setting 100a to "Test new patched".
* The biblios should change accordingly
* Check the Background jobs in the DB again:
* `select count(*) from background_jobs;`
* This should now be only 1 more then the last time (so eg 7)
* Check the details of the background jobs:
* `select json_extract(data,'$.record_ids'),enqueued_on from background_jobs order by enqueued_on desc limit 3;`
* The most recent entry should now contain three items in one row, and the following two should be the same as in the previous query (i.e. the old jobs from the edit before the patch)
+-----------------------------------+---------------------+
| json_extract(data,'$.record_ids') | enqueued_on |
+-----------------------------------+---------------------+
| [2299, 2300, 2301] | 2026-03-25 13:29:17 |
| [2300] | 2026-03-25 13:19:34 |
| [2299] | 2026-03-25 13:19:34 |
+-----------------------------------+---------------------+
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 1bb30a26cd)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Amended-by: Jonathan Druart
Tidy C4/Search.pm
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit b3cddc9994)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Testing:
- Follow plack intranet and/or opac error logs, tail -f /var/log/koha/kohadev/plack-*-error.log
- Login using web browser
- Do a POST request on either intranet or OPAC, for instance:
- Staff: Reports > create a new SQL report "select 'hello'"
- Public: search, select a book, add to a list
- 'uninitialized string' error appears in respective logs
- Apply patch
- restart_all
- Do another POST request
- No further errors should appear in the logs
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit f7327f6e45)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
This patch adds predefined values to the record_sources table on installation of a new Koha instance. The values are
- batchmod
- intranet
- batchimport
- z3950
- bulkmarcimport
- import_lexile
and were formerly hardcoded in marc-overlay-rules.tt. This was introduced in 35380 which took care of updated instances via a db_revs script but forgot installation scripts.
Testplan
a) start up a fresh ktd
b) login and check go to Administration -> Record sources
c) check that no source exists
d) apply patch
e) go to mysql shell 'koha-mysql kohadev'
f) empty the database to trigger Kohas installer: 'drop database koha_kohadev;' followed by a 'create database koha_kohadev;'
g) restart_all
h) open Koha in browser again, the web installer should appear now. Credentials are 'koha_kohadev' with pasword 'password'.
i) run through the installation and create an admin user,
j) login with that user and repeat b) and c)
k) check that record sources are now set with the values from the descriptioni above.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit d56976e9a6)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 9539b21089)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
This patch adds a new rule for itemtypes to allow/prevent filling other holds on a bibliographic record
The rule will obey ReservesControlBranch for determining the rules
To test:
1 - Apply patches
2 - Setup/choose 2 item types in Administration->Item types
3 - In Administration -> Circulation and fine rules scroll to the bottom
4 - Set one item type to "Fill other holds on record" = 'Yes' and the other to 'No'
5 - Place a next available hold on the record
6 - Checkout the 'Yes' item type to the patron
7 - Their hold should fill
8 - Check in the items
9 - Place a next available hold on the record
10 - Checkout the 'No' itemtype to the patron
11 - Their hold should not be filled
12 - Check in the item and confirm the hold
13 - Checkout the itme to the patron, the hold fills
NOTE: While seemingly counter intuitive, the rule controls whether a hold NOT tied to this item will fill
Confirming the hold adds an itemnumber to the hold. In most cases, these items will not be holdable, so won't trigger at checkin
14 - Set a rule for the patrons branch to No and set system preference ReservesControlBranch to 'Patrons library'
15 - repeat 5&6
16 - The hold should not fill
17 - Change reserves control branch to items home library
18 - checkin /checkout the yes item type
19 - The hold should fill
Signed-off-by: Andrew Auld <andrew.auld@openfifth.co.uk>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 876869544f)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
When logging a hold create action, include all hold fields that may be
defined at hold creation.
Fields that are not defined until later (such as found status, waiting
date, etc) are not included to avoid clutter.
To test:
1. System preference setup
i. Enable HoldsLog system preference
ii. Set AllowHoldPolicyOverride to Allow
2. Place a hold in the staff interface
3. View the action logs entry for that hold creation
i. Go to Tools > Log viewer
ii. Select Modules: Holds and Action: Create
iii. Click Submit
--> Note that the "info" column only shows the hold id
4. Apply patch and restart_all
5. Repeat steps 2-3
--> The "info" column should now show hold details
Signed-off-by: Emmanuel Bétemps <e.betemps@gmail.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
(cherry picked from commit 83e2c3888d)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
(cherry picked from commit 0d84b49ea7)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
(cherry picked from commit 6fbcedd5db)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
FixPriority has a read-modify-write structure: it SELECTs all active
holds for a biblionumber, computes new priority order in Perl, then
issues N UPDATEs. Two concurrent callers (e.g. two patrons adjusting
holds on a popular title at the same moment) could both read the same
snapshot and then overwrite each other's renumbering, leaving duplicate
or incorrect priority values.
Fix by:
1. Wrapping the entire function body in txn_do so it runs atomically.
txn_do uses MySQL savepoints when called inside an existing
transaction, so callers that already hold a transaction are unaffected.
2. Adding FOR UPDATE to the initial SELECT so the first caller
acquires row-level locks on all holds for that bib. A second
concurrent caller will block at the SELECT until the first commits,
then reads the already-corrected priorities and produces a consistent
result.
Note: FOR UPDATE locks only rows that exist at read time; a concurrent
INSERT of a brand-new hold can still slip through, but priority
assignment for new holds goes via AddReserve → FixPriority, which will
also acquire the lock and renumber correctly.
(cherry picked from commit e0bdf3e859)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
The lowestPriority rank-clamping block fired a second SQL query to get
max(priority) WHERE lowestPriority=0 and used that value as both a
threshold and a splice index. This was fragile: if priorities had
become discontiguous (e.g. due to a race or a bug elsewhere), the DB
priority value would not match the 1-based position in @priority, and
the splice could land the hold in the wrong slot.
The @priority array already contains all the information needed: it is
sorted lowestPriority ASC so the count of rows where lowestPriority=0
is exactly the number of non-lowest holds, which equals the last valid
rank for a normal hold. Fetch lowestPriority in the existing SELECT and
use scalar grep { !$_->{lowestPriority} } @priority instead.
This eliminates the extra DB round-trip and makes the clamping operate
on the same data snapshot that will be written back.
(cherry picked from commit 4d61ac803f)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
When FixPriority is called with a reserve_id but no rank (rank defaults to ''),
the empty string evaluates to 0 numerically, satisfying `$rank <=
$highest_non_lowest_priority`. This caused the clamping block to unexpectedly
set rank to a positive value, which then triggered an unintended splice and
moved the hold.
Add `$rank > 0` to the guard condition so the block only fires when an explicit
positive rank was requested, matching the pre-existing semantics of the splice
block below it.
(cherry picked from commit 4f32ba4323)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
The recursive self-call loop that ignoreSetLowestRank guarded against was
removed by the main patch. Remove the now-unused parameter from both the
function body and the POD documentation, and fix a double-word typo in the
POD while here.
(cherry picked from commit 8e8a5f914e)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
This patch removes the recursive calls that set each 'lowestPriority' hold to the bottom of the priority list
and replaces it them by simply ordering the list of 'remaining' holds by lowestPriorty, then priority.
A check is also added to ensure that if a lowestPriority hold is being adjusted, it cannot be moved above holds
not marked as lowest priority.
To test:
1 - Place many holds on a record, 50 - 150
https://github.com/kidclamp/handy-koha-script/blob/main/randhold.pl
This will randomly make some of the holds lowest priority
2 - View the record in the staff interface:
http://localhost:8081/cgi-bin/koha/catalogue/detail.pl?biblionumber=312
3 - Click on the 'Holds' tab
4 - Adjust the priority of several holds using the dropdowns
5 - Note the timing
6 - Apply the patch
7 - Adjust the priority of several holds using the dropdowns
8 - Confirm the timing is improved
9 - Confirm that holds adjust as expected
10 - Confirm you cannot move a lowest priority hold above a hold not marked lowest priority
11 - Confirm holds move correctly on a record where all holds are lowestPriority
12 - Confirm holds move correctly on a record where no holds are lowestPriority
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
(cherry picked from commit 1f089c41c1)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
This patch adds tests for the item_level and item_id parameters in the holds API.
It also updates the C4/Reserves.pm module to handle these parameters correctly.
Test plan:
1. prove t/db_dependent/api/v1/holds.t
Sponsored-by: Koha-Suomi Oy
(cherry picked from commit ee4f4f3357)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
This patch updates C4::Output::output_error to take
an additional argument which contains an "interface"
parameter, so that opac/tracklinks.pl can generate
an OPAC error instead of an Intranet error.
Test plan:
0. Apply patch and koha-plack --restart kohadev
1. Go to http://localhost:8081/test.pl
2. Note normal Intranet 404 error
3. Go to http://localhost:8080/cgi-bin/koha/tracklinks.pl?uri=https%3A%2F%2Ftest.com&biblionumber=29
4. Note normal Opac 404 error
4b. For extra points, note that tracklinks.pl returns a 404 instead of a 302 to the 404.pl page
5. Change 'TrackClicks' syspref to "Track"
6. Go to http://localhost:8080/cgi-bin/koha/tracklinks.pl?uri=https%3A%2F%2Ftest.com&biblionumber=29
7. Note normal Opac 404 error
7b. For extra points, note that tracklinks.pl returns a 404 instead of a 302 to the 404.pl page
8. Rejoice!
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
(cherry picked from commit 532d10a800)
Signed-off-by: Jacob O'Mara <Jacob.omara@openfifth.co.uk>
Martin Renvoize
David Cook
Sanjar Tulkinov Anvar o'g'li
Jonathan Druart
Kyle M Hall
Pedro Amorim
Hammat Wele
Marcel de Rooy
Andreas Roussos
Michael Hafen
Nick Clemens
Mark Hofstetter
Thomas Klausner
Petro Vashchuk
Eric Phetteplace
Jan Kissig
Lucas Gass
Emily Lamancusa
Andreas Jonsson
Johanna Räisä