For some reasons, with Search::Elasticsearch 8.0 JSON::true is
interpreted as 1 and Elasticsearch will always respond that it found "at
least 1 record". Something like this:
{
total => {
value => 1,
relation => 'gte'
}
}
Replacing JSON::true by \1 works with every version of
Search::Elasticsearch I tested (6.80, 7.717, 8.0)
Also worth noting:
Search::Elasticsearch will stop being supported by Elastic
https://github.com/elastic/elasticsearch-perl/issues/220
We might need to find an alternative for future versions of
Elasticsearch
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
Elasticsearch 8 removed the support of types. This patch adapts the
requests accordingly.
With this patch, Koha will still be compatible with Elasticsearch 7.x
but will no longer work with Elasticsearch 6.x
Since Elasticsearch 6.x is no longer maintained, this should not be a
problem.
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/removal-of-types.html
Tested with Search::Elasticsearch 6.80 and 7.717
Test plan:
1. Install Elasticsearch 8
(use docker image: docker.io/koha/elasticsearch-icu:8.x)
2. Change the elasticsearch server location in $KOHA_CONF
3. Run misc/search_tools/rebuild_elasticsearch.pl -r -b -v
It should fail
4. Apply the patch
5. Run misc/search_tools/rebuild_elasticsearch.pl -r -b -v
It should end with "Total XXX records indexed"
6. Try to search some biblios and verify that it works the same as
before
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
This patch adds debug logging to C4::SIP::Sip::MsgType::_initialize() to help
identify when critical fields (patron_id, item_id) are parsed as empty values.
Empty critical fields in SIP messages can later cause DBIx::Class constraint
errors when passed to Koha::Objects->find() operations, generating confusing
'Odd number of elements in anonymous hash' errors in SIP server logs.
The enhanced logging helps administrators identify the root cause by logging
when empty patron_id or item_id fields are detected during message parsing.
To test:
1. Apply regression tests patch first
2. Run:
$ ktd --shell
k$ prove t/db_dependent/SIP/MsgType.t
=> FAIL: Debug logging tests fail (no enhanced logging present)
3. Apply this patch
4. Run:
$ ktd --shell
k$ prove t/db_dependent/SIP/MsgType.t
=> SUCCESS: All tests pass, debug logging works correctly
5. Sign off :-D
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
These tests verify that C4::SIP::Sip::MsgType correctly parses SIP messages without
creating problematic arrayref field values, and validates enhanced debug logging for
empty critical fields.
Test Coverage:
• Message parsing with correct protocol versions (PATRON_INFO/HOLD/RENEW use v2, CHECKOUT
uses v1)
• Empty patron_id and item_id field detection with debug logging
• Malformed delimiter handling (double delimiters, trailing empty fields)
• Multiple consecutive empty fields in single message
• Confusing delimiter patterns that could break parsing
• Verification that parsed fields are strings, not arrayrefs
• Working siplog mock infrastructure in C4::SIP::Sip::MsgType
Arrayref Prevention Tests:
Tests verify message parsing DOESN'T create empty arrayrefs that would later cause "Odd
number of elements in anonymous hash" errors when passed to Koha::Objects->find()
operations.
Enhanced Debug Logging Tests:
Tests validate that empty critical fields (patron_id, item_id) are detected during
parsing and generate appropriate debug messages to help troubleshoot downstream
constraint errors.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
We should not allow anonymizing patrons with failed login attempts
when the pref FailedLoginAttempts has not been set.
Test plan:
Run the test again from the previous patch. This should pass now.
Choose a patron with a positive value (say 55) in borrowers.login_attempts
or modify one. (NOTE: This patron will be anonymized in the test plan.)
Empty pref value for FailedLoginAttempts.
Set PatronAnonymizeDelay to 1.
Fill dateexpiry for this patron to yesterday.
Run cleanupdatabase.pl -confirm -v and check if the patron was NOT anonymized.
* Anonymized 0 patrons
Now set FailedLoginAttempts to a value (say 55) so that the patron is
considered as locked. (This unrealistic value is for testing only.)
MAKE SURE that you will not be anonymizing more patrons than you like
in the next step! (Set login_attempts to value < 55 for all other patrons.)
Run cleanupdatabase.pl -confirm -v again and check if the patron was anonymized.
* Anonymized 1 patrons
Restore the changed pref values.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
Test plan:
Run the test without applying the follow-up patch.
The test should fail. This shows that we need the fix and will help
avoid future regressions.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
It has been removed in the controller by the previous patch.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
This patch forces download of invoice files rather than showing
them inline where they can potentially perform XSS
Test plan:
0. Apply the patch
1. koha-plack --restart kohadev
2. Find or create an active vendor
3. Create a basket if one doesn't already exist
4. Receive a shipment for the vendor
5. Create an invoice
6. Go to the invoice
7. In the URL change invoice.pl to invoice-files.pl
(or turn on AcqEnableFiles and go to it that way)
8. Upload an image file (e.g. SVG) or PDF
9. Click on the filename
10. Note that instead of displaying in the browser,
the file will be downloaded
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
Update the teams list to reflect the reality of the 25.11 cycle with an
updated release date and updates to the QA team to reflect those who
performed duties during the cycle.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
This change validates the displayby parameter in suggestion.pl
in order to prevent SQL injection.
Test plan:
0. Do not apply the patch
1. Log into the staff interface
2. Visit the path /cgi-bin/koha/suggestion/suggestion.pl?
op=else&displayby=status+AND+EXTRACTVALUE(1,CONCAT(0x7e,@@version))
3. Notice a 500 error is generated
4. In the error logs, the MySQL version number will appear
5. Apply the patch
6. koha-plack --restart kohadev
7. Visit the path /cgi-bin/koha/suggestion/suggestion.pl?
op=else&displayby=status+AND+EXTRACTVALUE(1,CONCAT(0x7e,@@version))
8. Notice that no error is generated
9. Try each of the "Organize by" options on the left nav bar
10. Note that none of them generate an error and that the option is
retained after clicking "Go"
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz>
This change updates the print format to use the built-in formatter
for operations like trim, strip HTML, strip newlines, and not
decoding entities.
Test plan:
0. Apply the patch
1. Go to http://localhost:8081/cgi-bin/koha/acqui/basket.pl?basketno=1
2. Click "add to basket"
3. Type "test" next to "From an existing record" and click "Submit"
4. On the "Gairm" row, click "Add order"
5. Fill in the required fields
6. In the "Internal note" put the following:
abc123
7. Click "Save"
8. In the Orders table, click "Export" and then "Print"
9. Note that the print page loads correctly, you can see the internal note,
and you cannot see lines for "Add internal note",
"Edit internal note", or "Add vendor note"
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
Merging patrons only required edit_borrowers permission
but would result in a patron deletion.
This patch requires a new merge_borrowers subpermission
To test:
1. Give a patron the following permissions
catalogue, edit_borrowers, list_borrowers
2. Log into the staff interface as the patron
3. Do a patron search and select 2 patrons
4. Click merge patron - success - a patron got deleted
5. Apply patch, updatedatabase and restart_all
6. Repeat steps 3 and 4 - you can't
7. Add the new permission for merge_borrowers
8. Repeat steps 3 and 4 - now you can
9. prove t/Koha/Auth/Patrons.t
Sponsored-by: CLAMS
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
Adding parentheses since I was not sure about precedence here. The filter
should be applied to the whole expression.
Adjusted double to single quotes in the second change.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
This change checks for HTTP(S) URLs and skips the shell when using
system() so that shell metacharacters aren't interpreted.
Test plan:
0. Apply the patch
1. Insert the following into the Babeltheque_url_update system preference:
$(whoami > /tmp/rce.log)
2. koha-shell kohadev
3. perl misc/cronjobs/social_data/update_social_data.pl
4. Note the script says the following:
$(whoami > /tmp/rce.log) is not a HTTP URL. Aborting.
5. Note also that /tmp/rce.log is not created
6. Change Babeltheque_url_update syspref to something like the following:
http://localhost:8080
7. perl misc/cronjobs/social_data/update_social_data.pl
8. Note that the wget succeeds but the bunzip2 fails (as expected) since
we haven't fetched a bzipped file
9. Try to create a payload that passes the HTTP URL test with shell
metacharacters and note that they're not interpetted and instead
just used as part of a URL string.
Signed-off-by: Jake Deery <jake.deery@openfifth.co.uk>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
This patch fixes an identified RCE in the update_social_data.pl
social_data, which when executed, allows an attacker to run arbitrary
code on the system's shell.
TO TEST:
a) In the Babeltheque_url_update syspref, wrap any bash in $(), and then
run the update_social_data.pl script.
*) For example, $(whoami >
/tmp/rce.log) will output to a file the username the perl script is
running as.
APPLY PATCH
b) Run the update_social_data.pl script again. The string is now
meta-escaped, and the RCE no longer works.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
This patch addresses CSV formula injection vulnerabilities by ensuring
all Text::CSV instances use formula protection.
Changes made:
* Add formula => 'empty' to all Text::CSV constructors in:
- Koha::Patrons::Import
The 'empty' formula mode causes potentially dangerous formulas (starting
with =, +, -, @) to be treated as empty strings rather than executed,
preventing CSV injection attacks while maintaining data integrity.
Security impact:
* Prevents execution of malicious formulas in CSV imports
* Protects against data exfiltration via CSV formula injection
* Maintains backward compatibility for legitimate CSV data
To test:
1. Apply the previous patch (author test)
2. Run:
$ ktd --shell
k$ prove xt/author/Text_CSV_Various.t
=> FAIL: Should show 1 file without formula protection
3. Apply this patch
4. Repeat 2
=> SUCCESS: All tests should now pass
5. Try importing CSV with formula content (=cmd|"/c calc",test,data)
=> SUCCESS: Formula content is rejected/emptied, not executed
6. Import normal CSV data
=> SUCCESS: Normal data imports correctly
7. Sign off :-D
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
This test ensures all Text::CSV usage in Koha includes proper formula
injection protection to prevent CSV formula injection attacks.
The test scans all Perl modules and verifies that Text::CSV constructors
include the 'formula' parameter set to a safe mode ('empty', 'die',
'croak', 'diag', or numeric 1-5).
To test:
1. Apply this patch
2. Run:
$ ktd --shell
k$ prove xt/author/Text_CSV_Various.t
=> FAIL: Test should fail showing files without formula protection:
- Koha/Patrons/Import.pm
3. Apply the follow-up patch to fix the violations
4. Repeat 2
=> SUCCESS: All tests should pass
5. Sign off :-D
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
This patch fixes a XSS vulnerability in the
suspend hold modal in the staff interface.
By building our Javascript using Web APIs rather
than with strings and jQuery's html() method,
we're able to securely handle malicious payloads.
Test plan:
0. Apply the patch
1. Go to http://localhost:8081/cgi-bin/koha/reserve/request.pl?biblionumber=29
2. Add a hold for "koha" patron
3. Go to http://localhost:8081/cgi-bin/koha/circ/circulation.pl?borrowernumber=51
4. Click "Holds" tab and click "Suspend"
5. Note the modal loads properly
6. Go to http://localhost:8081/cgi-bin/koha/members/moremember.pl?borrowernumber=51
7. Click "Holds" tab and click "Suspend"
8. Note the modal loads properly
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Bug 40538: Simplify code
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
See also comment33, point 4.
Test plan:
cp debian/templates/apache-shared-opac-antibot.conf /etc/koha/
Test enabling/disabling by toggling the comment of the SetEnvIf line,
restart apache and remove the KOHA_INIT and CGISESSID cookie.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
This patch adds a fast simple bot challenge via Apache.
To test:
0. Apply the patch
1. sudo cp debian/templates/apache-shared-opac-antibot.conf /etc/koha/.
2. sudo cp debian/templates/apache-shared-opac-plack.conf /etc/koha/.
3. curl http://localhost:8080/cgi-bin/koha/opac-search.pl
4. Note an instant response with a very simple HTML response
5. Open an incognito browser and go to http://localhost:8080/cgi-bin/koha/opac-search.pl
6. Note a very brief loading animation before the OPAC search page appearing
7. Do a search
8. Note no more loading animation appears
9. Close the incognito browser
10. Open an incognito browser and go to http://localhost:8080/cgi-bin/koha/opac-main.pl
11. Do a search via the "Library catalog" search bar
12. Note no loading animation appears
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Cornelius Hertfelder <cornelius.hertfelder@bsz-bw.de>
Signed-off-by: Magnus Enger <magnus@libriotech.no>
We have tested this in production on at least three live sites for
several weeks, and it has worked as expected. We have not noticed any
negative side effects.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
Test plan (only on a 22.11 !):
1 - Create a subscription, close it.
2 - Visit [YOUR INSTANCE]/cgi-bin/koha/serials/subscription-detail.pl?subscriptionid=[THE ID OF THE SUBSCRIPTION] -> there is a "close" button, it is not written closed at the right side of the title
3 - Apply patch
4 - repeat 2, there is no close button but a Reopen button + "(closed)"
is written after the title
Signed-off-by: David Flater <flaterdavid@gmail.com>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
This patch updates the teams.yaml to include the voted in 25.11
release team.
Test plan
1/ Check against https://wiki.koha-community.org/wiki/Release_Teams
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
We are able to comment a notice even when no user is connected
To test
1. Log in to OPAC.
2. Find a Biblio and open the comments tab.
3. Open another tab/window, and log out from the account.
4. Return to the first tab.
5. Click "Post your comments on this title."
--> We are able to post a comment
6. Apply the patch
7. Repeat step 1, 2, 3, 4, 5
--> You should be redirected to the login page.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Bug 13342: Tidy
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: wainuiwitikapark <wainuiwitikapark@catalyst.net.nz>
(cherry picked from commit 44d30ecc2c)