Commit Graph

5765 Commits

Author SHA1 Message Date
Pedro Amorim cb26be6d27
Bug 38751: Update Standard ILL backend migrate
Make use of newly updated ILL::Request->extended_attributes method instead of creating the query manually

The test plan in bug 38819 fails without this patch (page explodes with duplicate ID error).
It's the same cause as the one being fixed here in bug 38751 so I'm submitting this here.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 92fd65840a)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-22 23:01:37 +02:00
Pedro Amorim 2dfdc7c491
Bug 38751: Update Standard ILL backend add_request
Make use of newly updated ILL::Request->extended_attributes method instead of
creating the query manually

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 6526e00374)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-22 23:01:36 +02:00
Pedro Amorim 0b3f441166
Bug 38751: Update ILL::Request->extended_attributes
This updates the extended_attributes method to also act as setter.
Additionally, it'll skip adding any attribute that result in error,
preventing an ugly error 500.

Test plan:
1) Enable ILLModule sys pref
2) Attempt to create the following openurl request through the OPAC:
   http://localhost:8080/cgi-bin/koha/opac-illrequests.pl?DOI=10.1001%2Fjama.2024.11146&atitle=Interventions%20for%20High%20Body%20Mass%20Index%20in%20Children%20and%20Adolescents%3A%20US%20Preventive%20Services%20Task%20Force%20Recommendation%20Statement.&aulast=Nicholson%2C%20Wanda%20K.&backend=FreeForm&cardnumber=53807898&date=20240716&genre=article&id=DOI%3A10.1001%2Fjama.2024.11146&issn=00987484&issue=3&method=create&opac=1&openurl=1
3) Notice it fails with a Duplicate ID error and the OPAC blows up.
   This is because Koha is trying to add the 'DOI' illrequestattribute
   twice and that is resulting in a duplicate ID
4) Apply patches, repeat test plan, notice the error no longer happens,
   the request is created correctly and all successful attribute have
   been added.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 9fc470c0a8)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-22 23:01:35 +02:00
Jonathan Druart 809a6735d8
Bug 34068: Fix tests
Several wrong things in the previous attempt:
* as_list returns an array
* as_list returns objects, ->{library_id} does not exist
* as_list does not returns api object, you need to call ->branchcode to
  get the id
* $pickup_locations defined for both situations was confusing

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 44b29d114f)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-17 14:30:36 +02:00
Matt Blenkinsop a3a0d6dbb3
Bug 34068: Fix pagination for hold pickup locations
This patch addresses an issue where pagination filters are set incorrectly for the pickup locations endpoint. Only the final page is returned rather than the expected full list of results

Test plan:
1) In KTD, set the syspref RESTdefaultPageSize to 5
2) Set AllowHoldPolicyOverride to Allow
3) Create a hold on an item for a patron
4) Navigate to the patron's page and click on the holds tab
5) Click the dropdown in pickup locations
6) There will be two results, repeated three times
7) Apply patch and restart_all
8) Repeat step 5, there should now be a full list of 12 libraries

Co-authored-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 37f85baed4)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-17 14:30:35 +02:00
Martin Renvoize 1bb41e48ef
Bug 38423: Add unit tests
Sponsored-by: PTFS Europe <https://ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 210e0cb18b)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-17 14:30:35 +02:00
Martin Renvoize a269b1e295
Bug 38423: Code more defensively in process_invoice
We re-arrange the logic of process_invoice a little here to ensure we
skip order lines in invoices that do not have corresponding bib records.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit af19a0f553)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-17 14:30:35 +02:00
Kyle M Hall 5851316ff8
Bug 22415: (QA follow-up) Fix misc qa issues
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 7fb78e4a5d)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-17 14:30:35 +02:00
Martin Renvoize 318aa5103c
Bug 22415: Update EDI to use Koha::Logger
This patch updates the EDI code to use the Koha::Logger wrapper in
preference to the bare Log::Log4perl module.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 7f6c186831)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-17 14:30:34 +02:00
David Gustafsson 6fd5870d85
Bug 33268: Overlay rules don't work correctly when source is set to *
Fallback to overlay rules with wildcard filter if no match
found for exact filter match.

To test:
1. Administration => record overlay rules
2. Add this:
Module  Filter	                   Tag Preset  Added Appended Removed Deleted
Source  Z39.50 	                   245 Protect Skip  Skip     Skip    Skip
Source  * 	                   300 Protect Skip  Skip     Skip    Skip
Source  Batch record modification  245 Protect Skip  Skip     Skip    Skip
3. Add MARC modification templates:
Update existing or add new field 245$a with value CATSCATSCATSCATS
Update existing or add new field 300$a with value CATSCATSCATSCATS
4. Find a record that has those fields (likely any record would)
5. Use batch record modification on the record
6. 300$a should have been catified (expected, that's the bug)
7. 245$a is preserved
8. Change the value of the 300$a to something else.
   Temporarly disable MARCOverlayRules to be able to do so. And reenable
   it.
9. Apply the patch, restart services
10. Use batch record modification on the record
11. 300$a should be preserved, cat protection should have worked
12. Run tests: prove t/db_dependent/Biblio/MarcOverlayRules.t
13. Celebrate! :D

Sponsored-by: Gothenburg University Library
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Edit: tidied inline (tcohen)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jan Kissig <bibliothek@th-wildau.de>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit cb2d98fc5b)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-04-17 14:30:33 +02:00
Nick Clemens 358c60bee0
Bug 39236: writeoff_debts.pl needs adjustments
From bug 22435 - we need to use 'CREATE' as the account_offset type
From bug 31095 - we need to 'use Koha::Patrons;'

To test:
1 - Add a fine to a patron account
2 - perl misc/cronjobs/writeoff_debts.pl -v -c --added-after 2025-01-01
3 - Cronjob dies
4 - Apply patch
5 - Rerun
6 - Job succeeds!
7 - Confirm patron's fine is written off
8 - Sign off

Signed-off-by: Magnus Enger <magnus@libriotech.no>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 93e2a0953d)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:13:04 +01:00
Pedro Amorim f77d570e34
Bug 39175: Fix Attributes->Attribute
Bug introduced in FreeForm prior to adding it to core in this commit:
369dffb159 (diff-662970ddc6f6828afd23e6823399a4a32807c4a6c487a0aff86c9d116f60797dR806)

It should've been Koha::Illrequestattribute into Koha::ILL::Request::Attribute not Koha::ILL::Request::Attributes

To test, k-t-d:
1) Enable ILLModule syspref
2) Edit the only patron in k-t-d with the 'Interlibrary loan' category:
<staff_url>/cgi-bin/koha/members/memberentry.pl?op=edit_form&destination=circ&borrowernumber=16
3) Add a valid e-mail address to 'Primary email:'
4) Create a new ILL request:
/cgi-bin/koha/ill/ill-requests.pl?method=create&backend=Standard
5) Add any type, any valid patron cardnumber, any library. Click 'Create'.
6) Click 'Place request with partners'
7) Pick the only available partner library 'FRL-Walker'
8) Click 'Send email'.
9) Notice you get an error. Apply patch & restart plack. Repeat.
10) Confirm the there's no error and message was queue as expected:
$ select * from message_queue;

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 0d59a3db8d)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:13:03 +01:00
Andreas Jonsson 4bb59ec348
Bug 36025: Incorrect clause in patron search
This patch removes the search clause for
extended_attributes when there is no patron
attribute types which are searchable by default.

The clause was incorrectly added because the array
extended_attribute_types was rendered as [""]

This seems to be an issue with get_columns from DBIX Class and how that is cast when forcing an array
in the template. It returns an empty string for nothing, a scalar for a single value, and an array for multiple
values.

This patch now checks if the variable is set and has content, and wraps as an array if so. In the case of two attributes,
we do double the array, but this was true before as we always cast the return as an array.

Test plan:

* Make sure that no borrower attribute type is set to "searched by default"
  in Koha administration -> Patron attribute types.
* Enable the network monitring in the developer tools in your web browser
  (Ctrl + Shift + E in firefox or Ctrl + Shift + I and select the "Network"
  tab).
* Go to the patrons view members-home.pl (click patrons in top menu bar).
* Search for a string that will match multiple borrowers (to not be automatically
  redirected)
* Examine the ajax-request and make sure that "extended_attributes.code" is NOT
  part of the query.
* Also, in koha-testing-docker with selenium enabled,
  run 'perl t/db_dependent/selenium/patrons_search.t'
* Now set one patro attribute to searchable and searched by default and add a value for a patron, confirm search works
* Set a second patron attribute to searchable and searched by default and confirm searching works

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Works as advertised. Did not run the selenium tests, as they have
been obsoleted.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 208a7cc3a4)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:54 +01:00
Nick Clemens 4b29adb4bc
Bug 39282: Use generated MARC record to create item and return itemnumber
When creating items from the form we need to use the MARC record we build from the HTML->XML->MARC

This patch simply uses the variable we create and ensures we pass the itemnumbers back

To test:
1 - prove -v t/db_dependent/Koha/MarcOrders.t
2 - Still failing
3 - Apply this patch
4 - Success!

Phew! We did it.

Signed-off-by: John Vinke <jvinke@austinseminary.edu>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 3f6d482410)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:53 +01:00
Nick Clemens 89302013aa
Bug 39282: Clarify use of budget codes and ids
When getting items from the form on addorderiso2709.pl we are getting actual budget id's,
we only deal with codes when fetching the values from the MARC.
The code here assigns an id into a variable called code - this patch clarifies that and
ensures that we fallback to using the id if one has been passed from the interface form

To test:
0 - Apply first two patches
1 - prove -v t/db_dependent/Koha/MarcOrders.t
2 - It fails
3 - Apply this patch
4 - It fails?

Signed-off-by: John Vinke <jvinke@austinseminary.edu>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit fd609d9bb3)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:52 +01:00
Nick Clemens 21350cb20d
Bug 39282: Ensure items are created using Item Information values if not using MarcItemFieldsToOrder
When using MarcFieldsToOrder and not MarcItemFieldsToOrder it is expected that all of the item
information will come from the second tab 'Item information' on addorderiso2709.pl

Currently we are not and blank items are beign created

To test:
 1 - Apply unit tests
 2 - prove -v t/db_dependent/Koha/MarcOrder.t
 3 - It fails
 4 - Set MarcItemFields to Order:
     price: 949$p
     quantity: 947$c
     budget_code: 947$f
 5 - Ensure MarcItemFieldsToOrder is blank
 6 - Find a basket in acquisitions
 7 - Add to basket from a file
 8 - Fill item information, set 'Matching' at top of form to 'Do not look for matchign records' and save
 9 - Orders are added
10 - Click on a title
11 - Items do not load and an error is generated, this is because items don't have any values
12 - Check the DB:
     SELECT * FROM items WHERE homebranch IS NULL\G
13 - Note items from order are listed
14 - Apply the rest of the patches
15 - Repeat 6-10
16 - This time the items load successfully

NOTE: This patch alone doesn't address all errors, but subsequent will, keeping separate to address
each issue individually, but test plan can be executed as a whole against all

Signed-off-by: John Vinke <jvinke@austinseminary.edu>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 5034c27628)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:52 +01:00
Pedro Amorim 597a95d18b
Bug 38782: Remove concat in background process
The ud_provider->{report_type} is only defined on ->harvest_sushi, meaning it'll never be defined here.
We don't need this anyway. The filename should be the filename of the file being manually uploaded. We only need this concat logic for COUNTER files being generated by Koha following a SUSHI harvest, and that works.
This also gets rid of a warning when running the tests

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 0231911c98)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:32 +01:00
Pedro Amorim b08bc8fc60
Bug 36039: (QA follow-up): Improve audit messages
Improve messages clarity.
Show success message if no schema diff was found from audit.
Code refactor to ensure the title+message shown in UI is the same as the one shown in CLI and are all translatable.

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 7834d25fb4)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:31 +01:00
Pedro Amorim 1d6fb756f9
Bug 36039: (QA follow-up): Address i18n for print messages
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 42c1e109af)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:30 +01:00
Martin Renvoize dff1628187
Bug 36039: (follow-up) Move view logic to script/template
This patch updates the module as suggested by David to handle just the
diff code and leaves the display logic up to the templates and script.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: JesseM <jesse@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 534fba2691)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:29 +01:00
Pedro Amorim b7c7075dd4
Bug 36039: Move audit_database script logic to a dedicated module
Test plan #1 ensure same script functionality (same as bug 34064):
0. Apply patch
1. vi ./installer/data/mysql/kohastructure.sql
2. Comment out some columns, change NULL status, or whatever you like
3. perl misc/maintenance/audit_database.pl \
    --filename /kohadevbox/koha/installer/data/mysql/kohastructure.sql
4. Note that the output includes SQL commands to change the database
to match the new kohastructure.sql

5a. Try using koha-foreach and note that the database name appears above
the database comparison
5b. koha-foreach "perl misc/maintenance/audit_database.pl \
    --filename /kohadevbox/koha/installer/data/mysql/kohastructure.sql"

Test plan #2:
1. After doing test plan #1, access the new 'database audit' tab, visit:
<staff_url>/cgi-bin/koha/about.pl?tab=database
2. Ensure the output is the same as test plan #1

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: JesseM <jesse@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 09bb85409d)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:28 +01:00
Jonathan Druart 7eb1ed012c
Bug 37398: Do not rely on the columns names but data type instead
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit d2a6915010)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:25 +01:00
Lucas Gass 2d4b406742
Bug 37398: Make sure subfield and subfield_date are defined
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit d32218fb68)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:22 +01:00
Lucas Gass d2bfa731c8
Bug 37398: Make date and date time fields have date picker by default
To test:
1. APPLY PATCH, restart_all
2. Pick a framework, I choose BKS in k-t-d
3. Make sure that a few date and datetime fields are exposed in the editor.
4. On additem.tt a date picker should be instantiated on the date or datetime fields.
5. Make sure the date pickers work on each field.
6. If it is a datetime field you should have the option to add hours and minutes. ( datelastseen )
7. Edit the framework and remove the dateaccessioned.pl plugin from any of these date fields.
8. Even without the dateaccessioned.pl plugin a date picker should be present on the field.
9. Add the dateaccessioned.pl plugin to a non date/datetime field.
10. Make sure the dateaccessioned.pl plugin works properly on the non date/datetime field.
11. Mark one of the date/datetime fields as mandatory, make sure it still acts as mandatory.
12. Hide date/datetime from the editor, make sure it hides properly.

+----------------------+-------------+
| COLUMN_NAME          | COLUMN_TYPE |
+----------------------+-------------+
| dateaccessioned      | date        |
| replacementpricedate | date        |
| datelastborrowed     | date        |
| datelastseen         | datetime    |
| onloan               | date        |
+----------------------+-------------+

Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit aa4ee79f6e)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:21 +01:00
Pedro Amorim 868cfccf44
Bug 36627: (QA follow-up): Rename borrowernumber -> patron_id
To test:
$ prove t/db_dependent/api/v1/erm_counter_logs.t

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 15803e4f29)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:20 +01:00
Katrin Fischer b2a45e2e5f
Bug 36627: Fix typo "embeded"
A search turned up more of these in other files. Fixed as well.

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 3b86455e02)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:19 +01:00
Matt Blenkinsop 7265603c66
Bug 36627: Rename embed method to patron
This patch renames the embed method from borrowernumber to patron to fit with the use of 'patron' in the API

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 7b75586ead)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:16 +01:00
Matt Blenkinsop bd9b473567
Bug 36627: Embed patron record for display
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit da475321a5)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:15 +01:00
Matt Blenkinsop aa08e30cce
Bug 36627: Record who is importing sushi data
This patch adds functionality for storing the patron (or cronjob) that triggers a harvest. Currently every counter log is stored as being imported by the cronjob

Test plan:
1) In the usage statistics module in ERM, create a data provider (the details in the provider are not important)
2) Click into the provider and click into manual upload
3) Upload the file attached to this bug
4) Once the background job is completed click into the counter logs tab in the provider. The imported by columns should show a user with a clickable link

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 062e04c9c3)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 18:12:15 +01:00
Marcel de Rooy c374c92007
Bug 37784: (QA follow-up) Add few tables to Koha::Database::Columns
This resolves exceptions like:
Invalid table name accountlines at /usr/share/koha/C4/Reports/Guided.pm line 216.

Test plan:
Go to reports/dictionary.pl
Try to add a new definition for Serials.
Without this patch, you would crash (status 500).
Run t/Koha/Database/Columns.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-03-24 17:59:11 +01:00
Paul Derscheid 8ab34091c8
Merge remote-tracking branch 'upstream/24.11.x' into 24.11.x-security
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-26 19:24:37 +01:00
Jonathan Druart 59a2a2c80e
Bug 36081: Use multivalue_preference
C4::Context->multivalue_preference is not used so far and split on |
However the values of "multiple" sysprefs are separated by... comma!

Let support both here.

This patch also removes silly JS code in the template.

Signed-off-by: Magnus Enger <magnus@libriotech.no>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-24 20:43:20 +01:00
Marcel de Rooy ee6abf031f
Bug 36081: Check SupportedFormats server side
Test plan:
Add article request with format via OPAC.
Run t/db_dependent/Koha/ArticleRequest.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Tests in t/db_dependent/Koha/ArticleRequest.t pass. I can add an
article request with a type. If I allow PHOTOCOPY but change the
HTML in the OPAC form so SCAN is submitted I get a nice (but
somewhat generic) error.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-24 20:43:16 +01:00
David Cook 6d2b0c4a28
Bug 38488: Add Template::Toolkit filter for C4::Scrubber
This change adds a Template::Toolkit filter which is invoked
via " | scrub_html type => 'note' ".

Test plan:
0. Apply the patch
1. prove t/Koha/Plugins/HtmlScrubber.t

Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:20 +01:00
Tomas Cohen Arazi 2e0bcc4191
Bug 28478: Add Koha::Biblio->opac_suppressed()
This patch adds a convenient method at the Koha::Biblio level, as a
wrapper for the extrator added on this bug. Following the established
pattern we adopted a while back.

To test:
1. Apply this patch
2. Run:
   $ ktd --shell
  k$ prove t/db_dependent/Koha/Biblio.t
=> SUCCESS: Tests pass! All use cases covered!
3. Sign off :-D

Signed-off-by: Magnus Enger <magnus@libriotech.no>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:20 +01:00
Tomas Cohen Arazi b503cfe2be
Bug 28478: Add get_opac_suppression extractor
This patch adds an extractor method for the historically hardcoded field
942$n. This way we have a single place in which we code the extraction
and sanitization of its value.

To test:
1. Apply this patch
2. Run:
   $ ktd --shell
  k$ prove t/db_dependent/Koha/Biblio/Metadata/Extractor/MARC.t
=> SUCCESS: Tests pass! All use cases are covered!
3. Sign off :-D

Signed-off-by: Magnus Enger <magnus@libriotech.no>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:20 +01:00
Julian Maurice 551ab16e1f
Bug 38454: Flush memory cache before every API request
Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:19 +01:00
Lari Taskula 19cbf9701f
Bug 28907: REST - Drop support for allow-owner functionality
...and allow-guarantor functionality. Replaced by $c->auth->public($patron_id)
and/or $c->auth->public_guarantor($patron_id), where $patron_id is the patron's
id that owns the requested resource.

Old method, was applicable to both privileged and public routes:

  api/v1/swagger/paths/route.yaml

  x-koha-authorization:
    allow-owner: true
    allow-guarantor: true

New method, use public routes with no x-koha-authorization:

  GET /public/route/{patron_id}
  Koha/REST/V1/Controller#public_action:

  sub public_action {
      my $c = shift->openapi->valid_input or return;
      my $patron_id = $c->param( 'patron_id' );
      try {
          # Throws an exception that will render a response of 401 if not
          # authenticated and 403 if trying to access another user's resources
          $c->auth->public($patron_id); #or $c->auth->public_guarantor($patron_id)
          ...
          # other code
          ...
      }
      catch {
          $c->unhandled_exception($_);
      }
  }

  Another example of retrieving $patron_id when patron_id is not a request
  parameter:
  GET /public/another/object/{another_object_id}

  my $patron_id = Another::Object->find($another_object_id)->borrowernumber;
  try {
      # 403 if $another_object_id does not belong to API user
      $c->auth->public($patron_id);
      ...

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:18 +01:00
Lari Taskula 390814e215
Bug 28907: REST - Remove allow-owner from public article requests cancel route
To test:
1. prove t/db_dependent/api/v1/article_requests.t
2. Apply patch
3. prove t/db_dependent/api/v1/article_requests.t

Observe success in both cases.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:18 +01:00
Lari Taskula cb283f3b8e
Bug 28907: REST - Remove allow-owner from public checkout availability route
To test:
1. prove t/db_dependent/api/v1/checkouts.t
2. Apply patch
3. prove t/db_dependent/api/v1/checkouts.t

Observe success in both cases.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:18 +01:00
Lari Taskula 195136df34
Bug 28907: REST - Remove allow-owner from public patron hold cancellation
To test:
1. prove t/db_dependent/api/v1/patrons_holds.t
2. Apply patch
3. prove t/db_dependent/api/v1/patrons_holds.t

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:18 +01:00
Lari Taskula 48f8bdff0e
Bug 28907: REST - Remove allow-owner from public guarantors can see charges and checkouts
To test:
1. prove t/db_dependent/api/v1/patrons.t
2. Apply patch
3. prove t/db_dependent/api/v1/patrons.t

Observe success in both cases.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:18 +01:00
Lari Taskula 5d5bdf0415
Bug 28907: REST - Remove allow-owner from public checkouts route
To test:
1. prove t/db_dependent/api/v1/checkouts.t
2. Apply patch
3. prove t/db_dependent/api/v1/checkouts.t

Observe success in both cases.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:17 +01:00
Lari Taskula 24e08da733
Bug 28097: REST - Remove allow-owner from public password route
To test:
1. prove t/db_dependent/api/v1/patrons_password.t
2. Apply patch
3. prove t/db_dependent/api/v1/patrons_password.t

Observe success in both cases.

https://bugs.koha-community.org/show_bug.cgi?id=28907
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:17 +01:00
Lari Taskula c692261d4b
Bug 28907: Add Koha::REST::Plugin::Auth::PublicRoutes
To test:
1. prove t/db_dependent/Koha/REST/Plugin/Auth/PublicRoutes.t

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:17 +01:00
Lari Taskula 10e9863670
Bug 28907: Add REST exceptions for public routes auth
To test:
1. perl -c Koha/REST/Plugin/Exceptions.pm
2. perl -c Koha/Exceptions/REST.pm

More tests coming in following patches.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-18 19:17:17 +01:00
Tomas Cohen Arazi 8bef4aa38f
Bug 38726: (follow-up) Some more {Mm}arc ordering cases
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit f50978c256)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-13 10:04:50 +01:00
Janusz Kaczmarek af06851871
Bug 38913: (bug 38416 follow-up) Elasticsearch indexing explodes with oversized records
After Bug 38416 Elasticsearch indexing explodes with oversized
records, especially with UTF encoded data.

In Koha::SearchEngine::Elasticsearch::marc_records_to_documents a
following snippet has been introduced:

my $usmarc_record = $record->as_usmarc();
my $decoded_usmarc_record = MARC::Record->new_from_usmarc($usmarc_record);

But if $record is oversized (> 99999 bytes), it is OK for MARC::Record
object, but not for $record->as_usmarc. The produced ISO 2709 string
is not correct and hence cannot be properly converted back to
MARC::Record object by new_from_usmarc.

The result in this case can be like:

UTF-8 "\x85" does not map to Unicode at /usr/share/perl5/MARC/File/Encode.pm line 35.

Since it is done without any eval / try, the whole reindex procedure
(for instance rebuild_elasticsearch.pl) is being randomly interrupted
with no explanation.

Test plan:
==========
Hard to reproduce. But the explanation together with discussion in Bug
38416 (from 2024-12-15) explains and justifies the need of this added
eval.

1. Have a standard KTD installation with Elasticsearch.
2. Use the provided test record - add it to Koha with
   ./misc/migration_tools/bulkmarcimport.pl -b -file test.xml -m=MARCXML
   (have patience).
   During load process you should see a message like:
   UTF-8 "\xC4" does not map to Unicode at /usr/share/perl5/MARC/File/Encode.pm line 35.
3. The record should get biblionumber 439. Check in librarian interface with
   http://<your_addreess>:8081/cgi-bin/koha/catalogue/detail.pl?biblionumber=439
   that the record has been imported.
   However, you should not be able to make a search for this record.
4. Try to reindex with:
   ./misc/search_tools/rebuild_elasticsearch.pl -b -bn 439
   You should get a message like:
   UTF-8 "\xC4" does not map to Unicode at /usr/share/perl5/MARC/File/Encode.pm line 35.
   Again, no search results.
5. Apply the patch ; restart_all.
6. Repeat reindex with:
   ./misc/search_tools/rebuild_elasticsearch.pl -b -bn 439
   There should be no warning now and you should be able to find the record.

Signed-off-by: Magnus Enger <magnus@libriotech.no>
Followed the test plan. Works as advertised.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit c7127fabf4)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-04 21:52:48 +01:00
Tomas Cohen Arazi ce4522db14
Bug 38101: Make ES indexer split big fields into chunks
This patch makes the `_process_mappings()` method split the index values
in the event of them being bigger than the allowed 32766 bytes size.

To test:
1. Have KTD running with ES:
   $ ktd --proxy --es7 up -d
2. Perform a search
3. Pick the first result for edition
4. Find a cool Wiki page with lots of paragraphs
5. Copy all of the paragraphs and put them on a 500$a field for the record.
6. Repeat 2
=> FAIL: The record is not found
7. Reindex manually:
   $ ktd --shell
  k$ perl misc/search_tools/rebuild_elasticsearch.pl --biblios --where "biblionumber=3"  -v -v
=> FAIL: You get something like:
```
[22229] Committing final records...
One or more ElasticSearch errors occurred when indexing documents at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Indexer.pm line 148.
[22229] There were errors during indexing
Record #3 Document contains at least one immense term in field="note.raw" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[10, 109, 117, 115, 116, 97, 102, 97, 32, 102, 117, 101, 32, 101, 108, 32, 115, 101, 103, 117, 110, 100, 111, 32, 104, 105, 106, 111, 32, 100]...', original message: bytes can be at most 32766 in length; got 32771 (illegal_argument_exception) : max_bytes_length_exceeded_exception (bytes can be at most 32766 in length; got 32771)
[22229] Total 1 records indexed
```
8. Apply this patch
9. Repeat 7
=> SUCCESS: No error!
10. Repeat 2
=> SUCCESS: The record is indexed and can be found!
11. Sign off :-D

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 6998f5b7e1)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-04 21:50:22 +01:00
Tomas Cohen Arazi 5a8ebd6951
Bug 38678: Fix wrong attribute mapping in Koha::Old::Biblio
Because the mapping was defined in the wrong direction, it is not
possible to filter results in `GET /deleted/biblios` neither on the
`deleted_on` attribute, or even the fallback to DB fields when mappings
don't exist, on `timestamp`.

To test:
1. Apply the regression tests patch
2. Run:
   $ ktd --shell
  k$ prove t/db_dependent/api/v1/deleted_biblios.t
=> FAIL: A 200 returns a 500 instead
3. Apply this patch
4. Repeat 2
=> SUCCESS: Tests pass, behavior is correct
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit 5d59ec0baa)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
2025-02-04 21:40:55 +01:00