Commit Graph

55049 Commits

Author SHA1 Message Date
Lucas Gass bc2b96d1f5
Bug 41421: [24.05.x] Fix DB update
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-12-10 10:56:58 -07:00
Jesse Maseto fecd3d02f0
Update release notes for 24.05.16 release
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-12-04 19:22:07 +01:00
Jesse Maseto b7b5fb363c
Increment version for 24.05.16 release
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-12-04 19:22:06 +01:00
Paul Derscheid 8ea64e150f
Bug 35830: DBRev 24.05.15.001
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-12-04 19:22:06 +01:00
Brendan Lawlor 111a71085e
Bug 35830: Add merge_borrowers subpermission
Merging patrons only required edit_borrowers permission
but would result in a patron deletion.

This patch requires a new merge_borrowers subpermission

To test:
1. Give a patron the following permissions
    catalogue, edit_borrowers, list_borrowers
2. Log into the staff interface as the patron
3. Do a patron search and select 2 patrons
4. Click merge patron - success - a patron got deleted
5. Apply patch, updatedatabase and restart_all
6. Repeat steps 3 and 4 - you can't
7. Add the new permission for merge_borrowers
8. Repeat steps 3 and 4 - now you can
9. prove t/Koha/Auth/Patrons.t

Sponsored-by: CLAMS
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-12-04 09:59:00 +01:00
David Cook e5d5b050e1
Bug 40524: [24.05.x] Update print format to use built-in formatter
This change updates the print format to use the built-in formatter
for operations like trim, strip HTML, strip newlines, and not
decoding entities.

Test plan:
0. Apply the patch
1. Go to http://localhost:8081/cgi-bin/koha/acqui/basket.pl?basketno=1
2. Click "add to basket"
3. Type "test" next to "From an existing record" and click "Submit"
4. On the "Gairm" row, click "Add order"
5. Fill in the required fields
6. In the "Internal note" put the following:
abc123
7. Click "Save"
8. In the Orders table, click "Export" and then "Print"
9. Note that the print page loads correctly, you can see the internal note,
and you cannot see lines for "Add internal note",
"Edit internal note", or "Add vendor note"

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-12-04 09:58:57 +01:00
Fridolin Somers d7a06b7947
Bug 40525: [24.11] Fix missing use raw
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 69aadbad4b)
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-10-29 15:14:01 -04:00
Jesse Maseto 1248f17d82
generate release notes for 24.05.15
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-10-29 15:13:58 -04:00
Jesse Maseto 6cea33acd6
generate release notes for 24.05.15
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-10-29 15:13:56 -04:00
Jesse Maseto a1c889ae32
increment version for 24.05.15 release
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-10-29 15:13:54 -04:00
Lucas Gass dd17036139
Bug 40818: [24.05.x] Replace raw with html filter for marc_lib
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-10-29 15:13:52 -04:00
Jonathan Druart bcb301c513
Bug 40818: [24.05.x] Remove HTML from marc_lib
Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-10-29 15:13:20 -04:00
David Cook 4dee52666c
Bug 40525: [24.05] Update spreadsheet format to escape formulas
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-10-29 15:12:51 -04:00
Paul Derscheid 6ad5eeefea
Update release notes for 24.05.14 release
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-09-30 18:22:51 +02:00
Paul Derscheid 9e8c19ef03
Increment version for 24.05.14 release
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-09-30 18:22:50 +02:00
Lucas Gass 68ed159703
Bug 40766: [24.05.x] Prevent XSS on set-library
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-09-30 18:22:50 +02:00
Jake Deery 78b0a04d6a
Bug 40748: [24.05.x] Fix RCE in update_social_data.pl
This patch fixes an identified RCE in the update_social_data.pl
social_data, which when executed, allows an attacker to run arbitrary
code on the system's shell.

TO TEST:

a)  In the Babeltheque_url_update syspref, wrap any bash in $(), and then
run the update_social_data.pl script.
    *)  For example, $(whoami >
/tmp/rce.log) will output to a file the username the perl script is
running as.
APPLY PATCH
b)  Run the update_social_data.pl script again. The string is now
meta-escaped, and the RCE no longer works.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 40748: Check for HTTP URLs and skip shell when using system()

This change checks for HTTP(S) URLs and skips the shell when using
system() so that shell metacharacters aren't interpreted.

Test plan:
0. Apply the patch
1. Insert the following into the Babeltheque_url_update system preference:
$(whoami > /tmp/rce.log)
2. koha-shell kohadev
3. perl misc/cronjobs/social_data/update_social_data.pl
4. Note the script says the following:
$(whoami > /tmp/rce.log) is not a HTTP URL. Aborting.
5. Note also that /tmp/rce.log is not created
6. Change Babeltheque_url_update syspref to something like the following:
http://localhost:8080
7. perl misc/cronjobs/social_data/update_social_data.pl
8. Note that the wget succeeds but the bunzip2 fails (as expected) since
we haven't fetched a bzipped file
9. Try to create a payload that passes the HTTP URL test with shell
metacharacters and note that they're not interpetted and instead
just used as part of a URL string.

Signed-off-by: Jake Deery <jake.deery@openfifth.co.uk>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-09-30 18:22:46 +02:00
Jesse Maseto 193efe1142
Update release notes for 24.05.13 release
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-08-28 12:02:41 -04:00
Jesse Maseto 7ecc047dbd
Increment version for 24.05.13 release
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-08-28 12:02:41 -04:00
Tomas Cohen Arazi 443c067d30
Bug 40579: Fix CSV formula injection vulnerabilities
This patch addresses CSV formula injection vulnerabilities by ensuring
all Text::CSV instances use formula protection.

Changes made:
* Add formula => 'empty' to all Text::CSV constructors in:
  - Koha::Patrons::Import
  - Koha::BackgroundJob::ImportKBARTFile
  - Koha::ERM::EUsage::CounterFile

The 'empty' formula mode causes potentially dangerous formulas (starting
with =, +, -, @) to be treated as empty strings rather than executed,
preventing CSV injection attacks while maintaining data integrity.

Security impact:
* Prevents execution of malicious formulas in CSV imports
* Protects against data exfiltration via CSV formula injection
* Maintains backward compatibility for legitimate CSV data

To test:
1. Apply the previous patch (author test)
2. Run:
   $ ktd --shell
  k$ prove xt/author/Text_CSV_Various.t
=> FAIL: Should show 3 files without formula protection
3. Apply this patch
4. Repeat 2
=> SUCCESS: All tests should now pass
5. Try importing CSV with formula content (=cmd|"/c calc",test,data)
=> SUCCESS: Formula content is rejected/emptied, not executed
6. Import normal CSV data
=> SUCCESS: Normal data imports correctly
7. Sign off :-D

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-08-28 12:02:40 -04:00
Tomas Cohen Arazi a0ef38c4c3
Bug 40579: Add author test for CSV formula injection protection
This test ensures all Text::CSV usage in Koha includes proper formula
injection protection to prevent CSV formula injection attacks.

The test scans all Perl modules and verifies that Text::CSV constructors
include the 'formula' parameter set to a safe mode ('empty', 'die',
'croak', 'diag', or numeric 1-5).

To test:
1. Apply this patch
2. Run:
   $ ktd --shell
  k$ prove xt/author/Text_CSV_Various.t
=> FAIL: Test should fail showing files without formula protection:
   - Koha/Patrons/Import.pm
   - Koha/BackgroundJob/ImportKBARTFile.pm
   - Koha/ERM/EUsage/CounterFile.pm
3. Apply the follow-up patch to fix the violations
4. Repeat 2
=> SUCCESS: All tests should pass
5. Sign off :-D

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-08-28 12:02:40 -04:00
David Cook 81ef0aaccc
Bug 40538: Fix XSS in suspend hold modal
This patch fixes a XSS vulnerability in the
suspend hold modal in the staff interface.

By building our Javascript using Web APIs rather
than with strings and jQuery's html() method,
we're able to securely handle malicious payloads.

Test plan:
0. Apply the patch
1. Go to http://localhost:8081/cgi-bin/koha/reserve/request.pl?biblionumber=29
2. Add a hold for "koha" patron
3. Go to http://localhost:8081/cgi-bin/koha/circ/circulation.pl?borrowernumber=51
4. Click "Holds" tab and click "Suspend"
5. Note the modal loads properly
6. Go to http://localhost:8081/cgi-bin/koha/members/moremember.pl?borrowernumber=51
7. Click "Holds" tab and click "Suspend"
8. Note the modal loads properly

Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 40538: Simplify code

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-08-28 12:02:40 -04:00
Marcel de Rooy fa080e6b00
Bug 39906: (QA follow-up) Feature needs to be enabled specifically
See also comment33, point 4.

Test plan:
cp debian/templates/apache-shared-opac-antibot.conf /etc/koha/
Test enabling/disabling by toggling the comment of the SetEnvIf line,
restart apache and remove the KOHA_INIT and CGISESSID cookie.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-08-28 12:02:40 -04:00
David Cook 2f5428cb35
Bug 39906: Add a simple quick Apache-level bot challenge
This patch adds a fast simple bot challenge via Apache.

To test:
0. Apply the patch
1. sudo cp debian/templates/apache-shared-opac-antibot.conf /etc/koha/.
2. sudo cp debian/templates/apache-shared-opac-plack.conf /etc/koha/.
3. curl http://localhost:8080/cgi-bin/koha/opac-search.pl
4. Note an instant response with a very simple HTML response
5. Open an incognito browser and go to http://localhost:8080/cgi-bin/koha/opac-search.pl
6. Note a very brief loading animation before the OPAC search page appearing
7. Do a search
8. Note no more loading animation appears
9. Close the incognito browser
10. Open an incognito browser and go to http://localhost:8080/cgi-bin/koha/opac-main.pl
11. Do a search via the "Library catalog" search bar
12. Note no loading animation appears

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Cornelius Hertfelder <cornelius.hertfelder@bsz-bw.de>
Signed-off-by: Magnus Enger <magnus@libriotech.no>
We have tested this in production on at least three live sites for
several weeks, and it has worked as expected. We have not noticed any
negative side effects.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-08-28 12:02:39 -04:00
Martin Renvoize d077dcb01b
Bug 40022: [24.05.x] Add the 25.11 release team
This patch updates the teams.yaml to include the voted in 25.11
release team.

Test plan
1/ Check against https://wiki.koha-community.org/wiki/Release_Teams

Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-08-12 12:38:30 -04:00
Jesse Maseto 111f2d2031
Update release notes for 24.05.12 release
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-24 10:05:06 -04:00
Jesse Maseto 44237acacc
Increment version for 24.05.12 release
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-24 10:05:06 -04:00
Lucas Gass c8ac0e8255
Bug 39331: If patron category cant be guarantee do not continue with add_guarantors
To test:
1 - Remove age limits from a Child and Adult category in Administration->Patron categories. Ensure Child category can be guarantor and Adult cannot
2 - The above step is to avoid any age checks and form resubmission which removes the guarantor fields
3 - Find/create a child patron
4 - Add a guarantee from the Adult category to the child
5 - Check the DB:
   SELECT * FROM borrower_relationships WHERE guarantee_id={borrowernumber of child};
6 - Note the id colum
7 - Edit the child patron to Adult category
8 - Repeat 5
9 - Note the relationship still exists and the id is incremented
10 - Oof!
11 - APPLY PATCH, restart_all
12 - Try again steps 3 - 8
13 - The relationship should not be re-added
14 - Try with more than 1 guarantor on the record, confirming all are deleted
15 - Try adding some guarantor to patron types that should be able to, they relationship should work.
16 - Try changing some patrons to categories that do allow guarantors, making sure the relationship does stay.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-24 09:17:04 -04:00
Owen Leonard 9da8740955
Bug 38184: [24.05] (follow-up) Correct the number of tests
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-24 09:17:03 -04:00
Aleisha Amohia 2f8637325c
Bug 38184: [24.05] Show due date on OpacTrustedCheckout modal
This fixes checkout fields being made available to the public API.

Note: This depends on the fix at Bug 39313, so make sure you apply that
first.

To test:

1. Enabled OpacTrustedCheckout system preference
2. Search for an item and get the barcode
3. Go to the OPAC
4. Click the Self-checkout button at the top
5. Enter the barcode and submit
6. Notice that the due date does not show in the modal, even though the
   due date was correctly set

7. Apply the patch and restart services

8. Go to the staff interface and check in your barcode
9. Repeat steps 3-5
10. Confirm the due date does show correctly in the modal

11. Confirm tests pass t/db_dependent/Koha/Checkout.t

Sponsored-by: Reserve Bank of New Zealand
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-24 09:17:03 -04:00
David Cook 92ed47e182
Bug 39295: Prevent infinite loop on line wrapping
This change checks that the string width is larger than 0 before
attempting to perform line wrapping on the line.

Test plan:
0. Apply the patch and koha-plack --restart kohadev
1. Follow instructions at end of this message[1]
2. Note that the infinite loop does not happen and that a blank
card is created
3. Export the batch using the test layout and template that come
with koha-testing-docker
4. Note that the patron card is correctly created

[1]
To reproduce:

0. Change Koha test user's firstnameto "Firstname" and surname to "Lastname"

1. Create a template:
Units: PostScript Points
Page height: 60pt
Page width: 60pt
Card height: 60pt
Card with: 60pt

Top page margin: 10pt
Left page margin: 10pt
Number of columns: 1
Number of rows: 1
Gap between columns: 10pt
Gap between rows: 10pt

2. Create a layout:
Units: US inches

Field 1:
Text: Library Name
Font: Times-Roman
Font size: 12 pt
Text alignment: Left
Lower left X coordinate: 100in
Lower left Y coordinate: 100in

Field 2:
Text: <firstname> <surname>
Font: Times-Roman
Font size: 10pt
Text alignment: Left
Lower left X coordinate: 5in
Lower left Y coordinate: 5in

3. Create a batch for borrowernumber 51 (if using "koha" test user in ktd)

4. Export using the above template and layout

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-23 16:15:50 -04:00
Jonathan Druart 80edfd9607
Bug 39305: Adjust wording
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-23 16:15:50 -04:00
Jonathan Druart b264623e48
Bug 39305: Display a warning on the about page if Plack is not running
CSRF protection is only working if Koha is running under Plack.

The about page is not showing any errors if plack is not running.

Test plan:
0. Do not apply this patch
1. Go to the about page
=> Notice the "PSGI" entry in the "Server information" tab
2. Disable plack
koha-plack --disable kohadev && restart_all
3. Go to the about page
=> Notice that the "PSGI" entry is not displayed, and nothing indicates
that plack is not running
4. Apply this patch
=> "PSGI" now shows "Koha is running in CGI mode. This is a security issue! Use koha-plack --help"

Better wording idea welcome.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-23 16:15:49 -04:00
Tadeusz „tadzik” Sośnierz c7cf695279
Bug 39772: Extend background_tasks API to allow for human-readable job names
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-23 15:17:20 -04:00
Tadeusz „tadzik” Sośnierz a368406305
Bug 39772: Display package names for plugin jobs rather than "unknown" messages
This slightly improves the current situation, where on background_jobs.pl
each plugin job would be displayed as "Unknown job type 'plugin_...'".
After this patch, we display the implementing package name
(sourced from plugins' background_tasks()), which is slightly more
human-friendly and significantly less error-looking.

Test plan:

1. In KTD, install a plugin implementing background jobs (e.g. the KitchenSink)
2. On http://localhost:8081/cgi-bin/koha/plugins/plugins-home.pl,
   enable the plugin and run a tool/report from it
3. Observe ugly job types column on http://localhost:8081/cgi-bin/koha/admin/background_jobs.pl
4. Apply the patch
5. Observe the improved job types on http://localhost:8081/cgi-bin/koha/admin/background_jobs.pl

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-07-23 15:17:19 -04:00
Owen Leonard 1192a706cb
Bug 38127: Missing column headings in 'Add user' pop-up modal
This patch modifies the template for adding to a basket from a new
(empty) record in order to make the "Add user" patron search results
correct. The patron search results table headers must be defined in the
template before they are used.

To test, apply the patch and go to Acquisitions.

- Search for a vendor.
- Add or create a basket for that vendor.
- Click "Add to basket" and select "From a new (empty) record."
- In the "New order" form, under the "Patrons" heading, click "Add user"
- Search for patrons to add.
- In the patron search results, each column should have a heading: Card,
  Name, Category, Library.

Sponsored-by: Athens County Public Libraries
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>

	modified:   koha-tmpl/intranet-tmpl/prog/en/modules/acqui/neworderempty.tt
2025-07-07 13:14:39 -04:00
Nick Clemens 25dfb06f9b
Bug 39987: Correct breadcrumbs link to batch item deletion
This patch adds th del=1 param to the batchMod link in the breadcrumbs to ensure we return to batch item
deletion and not modification

To test:
1 - Browse to Cataloging->Batch item deletion
2 - Click the 'Batch item deletion' link in the breadcrumbs
3 - You are now at 'Batch item modification'
4 - Apply patch
5 - Repeat, you stay on the correct page
6 - Complete a batch item deletion and confirm the breadcrum is correct at each stage.

Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>

	modified:   koha-tmpl/intranet-tmpl/prog/en/modules/tools/batchMod-del.tt
2025-07-07 13:13:46 -04:00
Martin Renvoize 2e115a41b1
Bug 39887: Improve documentation for overdue_notices.pl
This patch corrects a few spelling mistakes and missing grammer and adds
a little more detail to the '--date', '--email' and '--frombranch'
options.

Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>

	modified:   misc/cronjobs/overdue_notices.pl
2025-07-07 13:12:51 -04:00
Jesse Maseto d8893bca20
Update release notes for 24.05.11 release
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>

	new file:   misc/release_notes/release_notes_24_05_11-1.html
	new file:   misc/release_notes/release_notes_24_05_11.md

Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-06-24 13:18:41 -04:00
Jesse Maseto 32c7f4d10d
Increment version for 24.05.11 release
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>

	modified:   Koha.pm
	new file:   installer/data/mysql/db_revs/240511000.pl

Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-06-24 13:18:40 -04:00
Tomas Cohen Arazi 7fbfc207ae Bug 38101: Make ES indexer split big fields into chunks
This patch makes the `_process_mappings()` method split the index values
in the event of them being bigger than the allowed 32766 bytes size.

To test:
1. Have KTD running with ES:
   $ ktd --proxy --es7 up -d
2. Perform a search
3. Pick the first result for edition
4. Find a cool Wiki page with lots of paragraphs
5. Copy all of the paragraphs and put them on a 500$a field for the record.
6. Repeat 2
=> FAIL: The record is not found
7. Reindex manually:
   $ ktd --shell
  k$ perl misc/search_tools/rebuild_elasticsearch.pl --biblios --where "biblionumber=3"  -v -v
=> FAIL: You get something like:
```
[22229] Committing final records...
One or more ElasticSearch errors occurred when indexing documents at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Indexer.pm line 148.
[22229] There were errors during indexing
Record #3 Document contains at least one immense term in field="note.raw" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[10, 109, 117, 115, 116, 97, 102, 97, 32, 102, 117, 101, 32, 101, 108, 32, 115, 101, 103, 117, 110, 100, 111, 32, 104, 105, 106, 111, 32, 100]...', original message: bytes can be at most 32766 in length; got 32771 (illegal_argument_exception) : max_bytes_length_exceeded_exception (bytes can be at most 32766 in length; got 32771)
[22229] Total 1 records indexed
```
8. Apply this patch
9. Repeat 7
=> SUCCESS: No error!
10. Repeat 2
=> SUCCESS: The record is indexed and can be found!
11. Sign off :-D

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-06-17 19:01:17 +00:00
Tomas Cohen Arazi 0c92fee632 Bug 38101: Regression tests
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Cook <dcook@prosentient.com.au>
2025-06-17 18:57:20 +00:00
Tomas Cohen Arazi a984f17434 Bug 37809: (follow-up) Allow embedding item.biblio too
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-06-13 15:30:07 +00:00
Tomas Cohen Arazi 3b2e93bf1c Bug 37809: (follow-up) Remove wrong embed
It seems to me like a copy and paste error coming from the holds
endpoint.

To test:
1. Use a REST API tool like Postman to ask for some patron's checkouts
2. Add x-koha-emebed: cancellation_requested
=> FAIL: It explodes, cannot embed it
3. Apply this patch
4. Repeat 2
=> SUCCESS: cancellation_requested no longer a valid option
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2025-06-13 15:26:00 +00:00
Tomas Cohen Arazi aa886d0f8d Bug 37809: Add more embed options to checkouts endpoints
This patch adds more embed options to checkouts endpoints.
To test you really need to try them on a REST tool.

* patron
* item
* library

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2025-06-13 15:26:00 +00:00
Nick Clemens 74e3696921 Bug 36736: Load plugins at the start of background job processing
This patch adds a call to get_enabled_plugins before processing background jobs to ensure
that all plugin hooks are loaded and cached

To test:
1 - Install a plugin that adds new objects e.g. the Contracts plugin
    https://github.com/bywatersolutions/fs-koha-plugin-contracts
    or the Koha Advent plugin:
    https://gitlab.com/koha-community/koha-advent/koha-plugin-fancyplugin
2 - Restart all
3 - Tail all your logs
4 - Stage and import a file containing items
5 - Note in the logs
    DBI Exception: DBD::mysql::st execute failed: Lock wait timeout exceeded; try restarting transaction
6 - Apply this patch
7 - Restart all
8 - Stage and import again
9 - Success!

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-06-09 18:54:43 +00:00
Jesse Maseto 82e4ca1288
Merge branch '24.05bug_37820' into 24.05.x 2025-06-09 13:00:16 -04:00
Emmi Takkinen 68e4aa4f48 Bug 37820: Upgrade fails at 23.12.00.023 [Bug 36993]
If table aqbudgets is miissing foreign key
'aqbudgetperiods_ibfk_1' database update fails on
"Can't DROP FOREIGN KEY" error.

To test:
1. Remove changes made in bug 32132, drop foreign key
aqbudgetperiods_ibfk_1 and downgrade your database:
- ALTER TABLE aqbudgets MODIFY COLUMN `budget_period_id` INT(11) NULL;
- UPDATE aqbudgets SET budget_period_id = NULL
WHERE budget_period_id IN(SELECT budget_period_id FROM aqbudgetperiods
WHERE budget_period_description = "Budget for funds without budget");
- DELETE FROM aqbudgetperiods
WHERE budget_period_description = "Budget for funds without budget";
- ALTER TABLE aqbudgets DROP FOREIGN KEY aqbudgetperiods_ibfk_1;
- UPDATE systempreferences SET value="23.1200022" WHERE variable = "Version;
2. Upgrade your database (e.g. running installer/data/mysql/updatedatabase.pl)
=> Update fails on error "Can't DROP FOREIGN KEY `aqbudgetperiods_ibfk_1`;...".
4. Apply this patch.
5. Try to update your database again.
=> Database should now be upgraded succesfully.
=> Confirm table aqbudgets now contains 'aqbudgetperiods_ibfk_1'.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-06-09 16:56:20 +00:00
Martin Renvoize 5b874c8f72 Bug 36978: (Bug 34029 follow-up) Make update idempotent
The update wasn't checking for index pre-existance and as such could
fail in certain cases.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jesse Maseto <jesse@bywatersolutions.com>
2025-06-09 16:48:50 +00:00
Paul Derscheid 0487fc3499
Update release notes for 24.05.10 release
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-05-26 22:31:36 +02:00