Commit Graph

3905 Commits

Author SHA1 Message Date
Marcel de Rooy d865ce1665 Bug 31250: (QA follow-up) The future will be easier
As requested by Jonathan, we need more flexibility ;)
Here it comes.

Test plan:
Run t/CookieManager.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit cffd36bcb4)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-31 20:45:24 +00:00
Marcel de Rooy 58eb3713ec Bug 31250: Deny clearing cookies with numeric suffix
This change allows us to add catalogue_editor_ to the deny list
in koha-conf.xml and keep cookies like catalogue_editor_123.

Test plan:
Run t/CookieManager.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

To test:
1 - Sign in to staff client
2 - Search for records and edit one
3 - Switch to advanced editor
4 - View cookies (F12/developer panel/storage tab)
5 - Note cookie like 'catalogue_editor_##' with value 'advanced'
6 - Log out
7 - note cookie value deleted
8 - Log in and search/edit a record
9 - Basic editor loads
10 - Apply patch
11 - Add line to koha-conf as described in second patch
12 - Restart all
13 - Switch to advanced editor
14 - Cookie value updated
15 - Logout
16 - Cookie value remains
17 - Log in and search/edit
18 - Confirm advanced editor loads

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit cb036faf2c)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-31 20:44:29 +00:00
Lucas Gass 83fea70a63 Bug 30483: DBIC
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-31 20:29:39 +00:00
Kyle Hall 0c6eef3816 Bug 31401: (QA follow-up) Remove Koha.ArePluginsEnabled, it is no longer needed
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit c5810c3605)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-31 20:14:27 +00:00
Jonathan Druart b71531cd58 Bug 32011: 2FA - Prevent qr_code to not be generated
Caught by selenium/authentication_2fa.t

 #   Failed test 'Enforce 2FA setup on first login'
 #   at t/db_dependent/selenium/authentication_2fa.t line 291.Error while executing command: unexpected alert open: Dismissed user prompt dialog: [object Object] at /usr/share/perl5/Selenium/Remote/Driver.pm line 411.
 at /usr/share/perl5/Selenium/Remote/Driver.pm line 356.

There is a 500 in the logs:
 "POST /api/v1/app.pl/api/v1/auth/two-factor/registration HTTP/1.1" 500

That is caused by
[ERROR] POST /api/v1/auth/two-factor/registration: unhandled exception (Mojo::Exception)<<Overflow error. version 8
total bits: 1268  max bits: 1232>>

It's coming from GD::Barcode, in my understanding we should increase 'Version'.

Test plan:
Remove all other subtests from the selenium to speed up its exec, then
run it in a loop. Without this patch the tests fail quite often (1/10),
now it should not fail with this error (maybe another one, see bug
32010)

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit edb16022fe)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-31 17:17:23 +00:00
Nick Clemens e207c09ee3 Bug 31421: Add a limited option to template plugin
This adds a specific 'limited' routine to the Categories plugin, and uses
it for the new patron dropdowns

To test:
1 - Limit a patron category to some branches
2 - Sign in not at one of those
3 - Confirm the new patron dropdowns don't include the category

Bug 31421: (follow-up) Add POD

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 1fc56e0e47)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-28 17:49:10 +00:00
Nick Clemens fbde769aec Bug 31421: Do not apply library limits to Categories plugin
This patch simply changes from using 'search_with_library_limits'
to 'search'

The limits I think are intended to prevent users from setting a patron to the wrong category,
they are not meant to hide patrons. Or rather, they don't do that, patrons are returned in search
but cannot be processed. The dropdowns as well won't contain the category, so you can see these
patrons, but not filter by the category

Bug 31422 and bug 31032 point out further issues with limits.

For now, let's fix searching

To test:
1 - Edit Patron category 'Patron/PT'
2 - Try to do a patron search that returns patrons in this category
3 - See 'Processing' and error in JS console: Uncaught TypeError: categories_map[data] is undefined
4 - Apply patch
5 - Restart all and repeate
6 - It works now!

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 29754226ea)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-28 17:48:50 +00:00
Marcel de Rooy 70c94b92ca Bug 31351: (QA follow-up) Use $self->json in Background modules
Making the disabling utf8 flag explicit instead of depending on
the default of the CPAN module.

Incorporating the change in background_jobs_worker too.

Test plan:
See next patches when we look at unit tests.
Restart koha-worker.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 202db89f83)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-14 21:35:57 +00:00
Joonas Kylmälä 144d6a24f0 Bug 31351: Koha::BackgroundJob: Let database connection object handle utf8 transcoding
Our database connections have been set up so that they will
automatically convert perl encoded strings to utf8 encoded strings
when sending data to database tables and decode the utf8 encoded
strings from the tables back to internal perl strings. As we can see
from a call in Koha/BackgroundJob.pm we are encoding the perl internal
string to utf8 and then decoding it back to perl internal string:

my $data_dump = decode_json encode_utf8 $self->data;

We can skip this unnecessary encode<->decode step (as the database
object has done the decoding for us) by simply calling the
JSON->new->decode() method which doesn't perform any string decoding.

Furthermore, the original code was buggy and didn't always remember to
encode the unencoded strings, in Koha::BackgroundJob::process we can
see

my $context = decode_json($self->context);

is missing the encode step. Now after this change encoding before
decoding is not necessary as we are using the methods from the JSON
module that do not perform any transcoding.

Note to those concerned whether the old data in the database is
compatible with this new code:
 Luckily our database connection object seems to be smart and
 didn't utf8 encode the utf8 returned data from the old encode_json()
 calls (probably checks the utf8 flag for the string (Encode::is_utf8($str))).

To test whether this fixes the original bug reported of not being able
to schedule background jobs with koha user having non-ASCII letters in
their surname:
 1) Change your staff users surname/lastname to "ääää"
 2) Log out and back in.
 3) Go to a biblio record detail page and click "Select all" in the
 items table
 4) Click Delete selected items and proceed with the deletion
 5) Notice the batch item deletion job has failed status
 6) Apply patch and repeat but this time the deletion job should finish.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit d95805c6aa)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-14 21:35:41 +00:00
Nick Clemens a72ecc3b27 Bug 31076: Handle year limits with equal sign or colon
To test:
1 - Enable ES and index
2 - Set expandedSearchOption to 'show'
3 - Advanced search on staff and opac using Year field on staff and Publication date range on OPAC for:
    2005
    2005-
    -2005
    <2005
    >2005
    2005-2010
4 - Confirm ranges don't work on staff side, but do on opac
5 - Apply patch
6 - Repeat searches - they work on both staff and opac
7 - prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit a11633379f)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-14 19:36:22 +00:00
Fridolin Somers 9d8ddcc1b7 Bug 28327: Unify CSV delimiter special behavior for tabulation
System preference 'CSVdelimiter' has a special case for tabulation.
Preference value contains string 'tabulation' but string '\t' must be used in CSV file.

This is OK in many places, for exemple Bug 17590.

This patch adds C4::Context->csv_delimiter to add a uniq metod dealing
with this behavior.
Also create Koha::Template::Plugin::Koha->CSVDelimiter for calls from
Toolkit Templates.

Test plan :
1) Set system preference 'CSVdelimiter' = 'tabs'.
2) Create CSV export in impacted pages
3) Check columns are separated by tabulation character and not string 'tabulation'
4) Check with another delimiter

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 381b79593e)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-10-14 19:29:41 +00:00
Nick Clemens 54ddc31c71 Bug 30779: Remove _update_import_record_marc and update tests
This patch removes the now unused: _update_import_record_marc

Additionally, as items are already present in import biblio we no longer need to embed
them, so that parameter is removed and the option removed from the sub and pod and everywhere
it was used

In all cases, we were embedding, so we don't need a way to get without

Tests updated

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b02b66acf5)
2022-10-03 22:48:43 +00:00
Tomas Cohen Arazi 3d3fc0f85a Bug 31245: Do not try to report results if task not started
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Andrew <andrewfh@dubcolib.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 12bc623ae7)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-30 22:47:51 +00:00
Nick Clemens 8b5dc95fe5 Bug 30879: Allow biblionumber as sort option in Elasticsearch
Repeat previous tests with Elasticsearch engine
You will need to reindex and reset mappings to pickup the changes form the file

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit c8c51867c4)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-29 16:47:07 +00:00
Marcel de Rooy 558ee09dd5 Bug 27849: (follow-up) Add test
Compacting the code a bit too.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit ad0aa754ee)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-29 15:41:54 +00:00
MJ Ray a48cf3c941 Bug 27849: Koha::Token may access undefined C4::Context->userenv
The _add_default_csrf_params internal function accesses
C4::Context->userenv without checking that it has been defined. I think
not all of the potential callers of it declare that they require a
defined userenv, so we should test and provide defaults for required
values if it is not defined, to avoid some "Can't use an undefined value
as a HASH reference" HTTP 500 Internal Server Errors.

To test:

Do anything that requires a form with CSRF token, such as editing your
details. Behaviour should be unchanged. To test the failure case, you
would need some customised code that indirectly generates a CSRF token
before setting the userenv up and I am not sure there is any in released
Koha yet.

Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>

Looks good to me. Working as expected.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 8c45fe1aaa)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-29 15:41:29 +00:00
Lucas Gass 2f2013d2da Bug 30984: DBIC
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-29 15:39:39 +00:00
Joonas Kylmälä b2a66cd307 Bug 31455: (QA follow-up) Make table creation O(N)
We can insert the indices before the main loop to a hash,
this way we don't have to look up during each loop iteration
the index from an array which in the worst case might take O(N)
thus making the total time complexity O(N^2).

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 223d9d3a99)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-16 15:21:55 +00:00
Lucas Gass d8956d387d Bug 31455: Make batchMod sort by order scanned
To test:
1. In batch item deletion and batch item modification add some
   barcodes.
2. Notice they are being sorted by item number, not in the order
   scanned.
3. Appply patch and restart services.
4. Try scanning items again in both batch item deletion and batch item
   modification.
5. The found barcodes should now sort by order scanned.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 5c2b2ac09a)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-16 15:21:38 +00:00
Tomas Cohen Arazi e8405efbec Bug 31223: Do not explode if plugins disabled
This patch makes the code skip calculating the plugin-related task
codes => class mapping if plugins are entirely disbaled.

To test:
1. Apply the regression tests
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Plugins/BackgroundJob.t
=> FAIL: Tests explode with: Can't call method "GetPlugins" on an
undefined value at... Koha/BackgroundJob.pm line 424.
3. Apply this patch
4. Repeat 2
=> SUCCESS: Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 7867106d43)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-12 14:56:37 +00:00
David Cook 9244190804 Bug 15187: Index 880 in Zebra the same as Elasticsearch
This patch modifies the MARC21 export to Zebra, so that 880 fields
are rewritten as their linked fields, in the same way that we
already do with Elasticsearch, so that the alternate graphic
representation of fields are indexed accordingly. (ie 880 $6245-01
Chinese titles will be indexed into the title index using the 245 rules)

Test plan:
0. Apply patch
1. Turn on ICU indexing
1b. vi /etc/koha/zebradb/etc/default.idx
1c. Replace charmap word-phrase-utf.chr with icuchain words-icu.xml
1d. Replace charmap word-phrase-utf.chr with icuchain phrases-icu.xml
1e. Restart Zebra server
1f. Re-index Zebra
2. Add record with a 880 $6 245-01 $a 教牧書信 field.
3. Search for this record using a title index with the Chinese title
4. Note that the record is correctly retrieved

(Note: This test probably works better using author or series as they
 present as links on the detail page which makes the fix more obviously
 useful.)

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 85690a14f8)
2022-09-02 17:56:24 +00:00
Lucas Gass 0fd448f77a Bug 31086: DBIC
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-02 17:12:24 +00:00
Nick Clemens e03b542a18 Bug 31086: Throw an exception when a hold is stored with no branchcode
To test:
1 - Apply first 2 patches
2 - On staff client, find a biblio, place hold, select a patron
3 - Before creating hold, right click on 'Pickup location'
4 - Inspect the element
5 - In the console find the top <li> for pickup, and right click
6 - Select 'Delete node'
7 - Save hold
8 - You get an exception/error

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit e7d886d388)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-02 17:07:18 +00:00
Kyle Hall 73f7c89ba6 Bug 30458: Include librarian (manager_id) in accountline when using "Payout amount" button
When using the "Payout amount" button, accountlines.manager_id is not
populated. This is not the case with the "Issue payout" buttons.

Test Plan:
1) Create a manual credit
2) Issue a payout using the "Payout amount" button
3) Inspect the payout accountline, note it has no manager_id
4) Apply this patch, restart all the things!
5) Repeat steps 1 and 2
6) Note the new payout accountline has a manager_id!

Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
JK: Fix commit title

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 0177b88eb3)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-09-02 14:56:35 +00:00
Nick Clemens 3f7363f6ca Bug 30447: Check if transfers arrived or cancelled
TO test:
 1 - Find a bib with a single item
 2 - Browse to Circulation->Transfer
 3 - Transfer the item to another branch
 4 - Check the item in at current branch and cancel the transfer on the dialog
 5 - Place a hold on the biblio for a patron at current branch
 6 - Browse to Circulation->Holds to pull
 7 - Item does not show
 8 - Apply patch, restart all
 9 - Browse to Circulation->Holds to pull
10 - Item shows

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 206ba015c4)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 19:34:49 +00:00
Shi Yao Wang 73d5fd08f5 Bug 29922: Make get_search_groups return results sorted by title
get_search_groups from Koha/Library/Groups.pm returns library groups sorted by title.

Test plan:
1- Go to administration > library groups and create multiple library groups with different names (e.g. AGroup, BGroup, ÄGroup, etc.). Make sure to tick "Use for OPAC search groups" checkbox.
2- Go to advanced search in OPAC.
3- Under "Groups of librairies" , click the select button.
4- Notice that the library groups are not in alphabetical order.
5- Apply the patch.
6- Notice that the library groups are in alphabetical order.
7- Run `prove t/db_dependent/LibraryGroups.t` and all test should pass.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit d86d949551)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 19:32:30 +00:00
Martin Renvoize f00a28b576 Bug 29105: Add effective_itemtype handling to items api
This patch adds effective_item_type_id to the items api.

We overload the to_api method to append the effective_item_type_id field
with that of the effective itemtype code.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit dc75b5d5cd)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 19:19:03 +00:00
Lucas Gass 1ae24dbb18 Bug 29632: Schema update
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 17:52:46 +00:00
Nick Clemens 256ef4ee27 Bug 29632: Don't sort cn-sort numerically
When defining our sort fields in we defined all as 'numeric'

For other string containing numbers this is likely correct, however,
for callnumbers it is not. e.g. E45 should sort before E7

This patch adds a new 'callnumber' type and deifnes this for cn-sort and
adds to the field maping a sort without numeric set

To test:
0 - Be using ES with Koha
1 - On records with single item, add callnumbers:
    VA65 E7 R63 1984
    VA65 E7 T35 1990
    VA65 E45 R67 1985
2 - Add public note 'shrimp' or something to make them easily searchable as a group
3 - Search for 'shrimp', sort by callnumber
4 - Note E45 comes last, it should come first
5 - Apply patch
6 - Reset ES mappings
7 - Reindex ES
8 - Repeat search
9 - Sorting should be correct when set to callnumber

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Michal Urban <michalurban177@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b8b8a62f11)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 17:35:16 +00:00
Martin Renvoize c3c889e7f4 Bug 31036: Treat SIP00 that same as CASH
SIP00 is the code for CASH transactions that have taken place on a SIP
client.  Cash management treats CASH type transactions as a special case
for banking and register requirements. This patch brings SIP00 in line
with the CASH handling code of cash management.

Test plan
1) Enable cash management with 'EnablePointOfSale'
2) Enable cash registers with 'UseCashRegisters'
3) Add a cash register via 'Administration > Cash registers'
4) Make some payments against the ash register in the staff client
5) Attempt to make a payment via a SIP client without having associated
   a cash register to the SIP account config
5a) This should now fail
6) Attach the cash register to the SIP account used above
6a) Payment should now work again on SIP transactions
7) Look at the cash register details page for the cash register
7a) 'Bankale' should now include cash from both SIP and Staff client
sales

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 9d8be887d0)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 17:20:17 +00:00
Tomas Cohen Arazi e9fde507b6 Bug 31069: Remove ExplodedTerms dependency on templates
This patch removes the use of templates and CGI in ExplodedTerms by using the
Koha::I18N library.

The functionality on the package is too simple, and messing with the
template paths complexity on the tests was too much, given we have a
nice way to have translatable strings at the package level.

This patch does that, and cleans up the test file as well, that required
complex template and CGI mocking to run properly.

To test:
1. Run:
   $ kshell
  k$ prove t/db_dependent/SuggestionEngine_ExplodedTerms.t
=> FAIL: It fails in master
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests pass
4. Follow the original test plan
=> SUCCESS: Things work
5. Enjoy the rest of the day and forget this ever happened

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit ecd91d6089)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 17:15:18 +00:00
Kyle Hall 20a9c16e6b Bug 31054: Manual importing for EDIFACT invoices fails with a 500 error page
Error in plack log is: Can't locate object method "new" via package
"Koha::Plugin::Com::ByWaterSolutions::MyEdifactPlugin" (perhaps you
forgot to load "Koha::Plugin::Com::ByWaterSolutions::MyEdifactPlugin"?)
at /usr/share/koha/lib/Koha/EDI.pm line 219.

Test Plan:
1) Set EdifactInvoiceImport to "Don't"
2) Configure an EDI vendor to use an EDIFACT plugin
3) Attempt to import an invoice file
4) You will be shown an error page
5) Apply this patch
6) Restart all the things!
7) Attempt to import another invoice file
8) It works!

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 181b0c16fe)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 15:49:52 +00:00
Tomas Cohen Arazi 7e58a301b5 Bug 29333: (QA follow-up) POD consistency
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 04b6ed16c6)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 15:30:27 +00:00
Julian Maurice 76e980bb1a Bug 29333: Fix encoding of imported UNIMARC authorities
MARC::Record and MARC::File::* modules sometimes use the position 09 of
the leader to detect encoding. A blank character means 'MARC-8' while an
'a' means 'UTF-8'.

In a UNIMARC authority this position is used to store the authority type
(see https://www.transition-bibliographique.fr/wp-content/uploads/2021/02/AIntroLabel-2004.pdf [FR]).
In this case, 'a' means 'Personal Name'.

The result is that the import will succeed for a Personal Name
authority, but it will fail for all other authority types.

Steps to reproduce:
0. Be sure to have a Koha UNIMARC instance.
1. Download the MARCXML for "Honoré de Balzac"
   curl -o balzac.marcxml https://www.idref.fr/02670305X.xml
2. Verify that it's encoded in UTF-8
   file balzac.marcxml
   (should output "balzac.marcxml: XML 1.0 document, UTF-8 Unicode
   text")
3. Go to Tools » Stage MARC for import and import balzac.marcxml with
   the following settings:
   Record type: Authority
   Character encoding: UTF-8
   Format: MARCXML
   Do not touch the other settings
4. Once imported, go to the staged MARC management tool and find your
   batch. Click on the authority title "Balzac Honoré de 1799-1850" to
   show the MARC inside a modal window. There should be no encoding
   issue.
5. Write down the imported record id (the number in column '#') and go
   to the MARC authority editor. Replace all URL parameters by
   'breedingid=THE_ID_YOU_WROTE_DOWN'
   The URL should look like this:
   /cgi-bin/koha/authorities/authorities.pl?breedingid=198
   You should see no encoding issues. Do not save the record.
6. Import the batch into the catalog. Verify that the authority record
   has no encoding issue.
7. Now download the MARCXML for "Athènes (Grèce)"
   curl -o athènes.marcxml https://www.idref.fr/027290530.xml
8. Repeat steps 2 to 6 using athènes.marcxml file. At steps 4 and 5 you
   should see encoding issues and that the position 9 of the leader was
   rewritten from 'c' to 'a'. Strangely, importing this batch fix the
   encoding issue, but we still lose the information in position 09 of
   the leader

This patch makes use of the MARCXML representation of the record instead
of the ISO2709 representation, because, unlike
MARC::Record::new_from_usmarc, MARC::Record::new_from_xml allows us to
pass directly the encoding and the format, which prevents data to be
double encoded when position 09 of the leader is different that 'a'

Test plan:
- Follow the "steps to reproduce" above and verify that you have no
  encoding issues.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 01d78e1ec7)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-23 15:30:11 +00:00
Lucas Gass f48fbde58e Bug 24010: Schema changes
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-12 15:35:47 +00:00
Tomas Cohen Arazi 2e063b708d Bug 31106: Make clean_search_term escape double quotes
We noticed that several characters will break Zebra queries. So search
terms need to be quoted for things to work. In this context, double
quotes inside search terms are problematic because double quotes are
what we use for quoting strings.

This patch makes the clean_search_term method escape double quotes.

To test:
1. Apply the unit tests patch
2. Run:
   $ kshell
  k$ prove t/Koha/SearchEngine/Zebra/QueryBuilder.t
=> FAIL: It doesn't work as it should!
3. Apply this patch
4. Repeat 2
=> SUCCESS: It does the job!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 24bb15785d)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-12 14:39:18 +00:00
Nick Clemens b76ef70469 Bug 31179: (QA follow-up) POD fix
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 0596405660)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-12 14:29:48 +00:00
Jonathan Druart 210a62cc3d Bug 31179: Don't copy invisible subfields when duplicating items
Duplicate item is intended to duplicate the visible cataloging fields of an item, however,
currently it is duplicating the complete internal record of the item

To recreate:
1 - find an item in Koha staff client, copy the barcode
2 - Issue this item to a patron
3 - Return to the record
4 - Edit items
5 - Click 'Actions->Duplicate' for the item in question
6 - Save the item
7 - Note in the items table above for that 'Total checkouts' 'Due date'
 etc. have not been copied to new item

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit b90787d5c8)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-12 14:29:34 +00:00
Martin Renvoize e0fcd54a6e Bug 29958: Do not set dateaccessioned on updates
This patch reomves the second occurence of setting daeaccessioned today,
outside of the 'add/update' handling in Koha::Item->store.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 1f5fbcf736)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-09 22:07:20 +00:00
Kevin Carnes d6a3a10fc7 Bug 30152: Elasticsearch - queries with OR don't work with limits
When a query with "OR" is combined with a limit in Elasticsearch, the precedence is not preserved and the results are not correct.

To test:
1) Set SearchEngine to Elasticsearch
2) Index records in Elasticsearch
3) Do an advanced search
4) Select More options
5) Enter a value for the first Keyword (e.g. Novels)
6) Change "and" before the second Keyword to "or"
7) Enter another value for the second Keyword (e.g. Prose)
8) Limit the search (e.g. Item type Books)
9) Do the search
10) Observe that records with the first keyword are not in the results
11) Apply the patch
12) Repeat the search
13) Observe that results with both keywords are in the results
14) Sign off

Sponsored-by: Lund University Library

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
(cherry picked from commit 3ed649a0e0)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-09 22:03:17 +00:00
Aleisha Amohia ec40a9c826 Bug 30823: Filling recalls uses 'FILL' action in action logs
This enhancement changes recall fulfillment actions to log with the
FILL action. It will also update existing recalls FULFILL actions in
the database to use the FILL action.

To test:
1) Enable the UseRecalls system preference and set up your
recalls-related circulation rules. Confirm RecallsLog is enabled.
2) Check out an item to Patron B.
3) Log into the OPAC as Patron A and search for the item.
4) Place a recall on that item.
5) Go back to the staff client and check the item in. Confirm the recall
as waiting for Patron A.
6) Check out the item for Patron A to fill the recall.
7) Go to Tools -> Log Viewer. Confirm there is a FULFILL action. Choose
the following search params to browse system logs:
- modules: recalls
- actions: fill
8) Submit the search and confirm the recall DOES NOT show.
9) Apply the patch, update database, restart services.
10) Refresh the log viewer and repeat step 7. Submit the search and
confirm the recall DOES show. Confirm there is no longer a FULFILL
action as both holds and recalls will use FILL.
11) Check in the item.
12) Repeat steps 2-6. We are ensuring that future recalls are logged
using the FILL action.
13) Repeat step 7. Confirm all test recalls are now showing in search
results.

Sponsored-by: Catalyst IT

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 7b8939b2ec)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-08-09 21:20:48 +00:00
Tomas Cohen Arazi 484add49de Bug 30889: (follow-up) Warn if context is not defined
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit de90b39b4c)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-07-29 17:25:21 +00:00
David Cook 2b4eb64563 Bug 30865: Double-quote Host-item in Koha::Biblio->get_components_query
This patch adds double quotes around the term qualified by "Host-item"
in Koha::Biblio->get_components_query().

Without them, reserved charactrs like "=" will cause syntax errors like
"CCL parsing error (10014) Unknown qualifier ZOOM for query:
    Host-item=(MyTitle = Mysubtitle)
    at /usr/share/koha/lib/C4/Search.pm line 245."

Test plan:
0) Don't apply the patch
1) Create biblio with title and subtitle like (MyTitle : MySubtitle)
2) Note the warning "There was an error searching for analytic records,
   please see the logs for details." on the detail page
3) Apply the patch
4) koha-plack --restart kohadev
5) Refresh the detail page
6) Note that the warning message is gone
7) prove t/db_dependent/Koha/Biblio.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit fd385d90b8)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-07-29 17:23:01 +00:00
Martin Renvoize 248dc2787e Bug 30744: Use RecordProcessor in get_marc_notes
This patch utilises RecordProcessor to filter the MARC::Record for the
right interface prior to constructing the marc notes array.  We also
remove the use of C4::XSLT for replacing AV values in the MARC fields in
preference to using the RecordProcessor filter.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 029d7bacf0)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-07-29 17:21:23 +00:00
Martin Renvoize 2ad8c15dbf Bug 30918: Allow passing filtered record to get_marc_notes
This patch does the absolute bare minimum to prevent private notes from
appearing on the OPAC.

Test plan
1. Go to Koha Administration -> Koha bibliographic frameworks
2. View the MARC structure for your BKS framework (or something else)
3. Search for tag 583, edit subfields
4. Go to subfield 'x' - nonpublic note. Confirm the OPAC visibility
   checkbox is UNCHECKED.
5. Edit or create a record using the BKS framework. Put a note in the
   583$x.
6. View this record in the OPAC
7. Go to the Title notes tab. Confirm the non-public note is
   showing, even though the framework says it should not be
   visible via the OPAC.
8. Apply patch
9. Confirm the non-public note is no longer visible

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 0efae9ed05)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-07-29 17:19:15 +00:00
Martin Renvoize 965e985619 Bug 30848: Fix issue exposed by unit tests
This patch changes the 'map' to a simple for loop so that we can easily
map multiple subfields to a field without overwriting the first previous
subfields in the structure.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 6c381c4b22)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-07-29 17:15:58 +00:00
Martin Renvoize 0a86104294 Bug 30848: Add an ExpandCodedFields RecordProcessor filter
This patch introduces a RecordProcessor filter for MARC::Record objects
that replaces Koha codes with descriptions in the MARC::Record passed to the processor.

Target usage:

  my $biblio = Koha::Biblios->find(
      $biblio_id,
      { prefetch => [ metadata ] }
  );

  my $record = $biblio->metadata->record;

  my $processor = Koha::RecordProcessor->new(
    {
        filters => ('ExpandCodedFields'),
        options => {
            interface     => 'opac',
            frameworkcode => $biblio->frameworkcode
        }
    }
  );

  $processor->process( $record );

Test plan
* Read the included unit test and confirm it makes sense and passes

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 62bba6d9e1)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-07-29 17:15:30 +00:00
Lucas Gass 770b9a0757 Bug 30889: Update Scheme(DBIC)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-07-29 17:08:00 +00:00
Martin Renvoize 671fbd4c15 Bug 30889: (follow-up) Record and use context in background_jobs
This patch records the current context to the background_jobs table at
enqueue time and then uses that record to set the context at process
time.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit bde3a32277)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-07-29 17:05:26 +00:00
Marcel de Rooy 7f64635a66 Bug 31053: Add Context module to Koha::Encryption
Test plan:
perl -MKoha::Encryption -e'print Koha::Encryption->new->encrypt_hex("test");'

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 147f6e0293)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2022-07-29 16:42:41 +00:00