Commit Graph

3397 Commits

Author SHA1 Message Date
Martin Renvoize 33165bd22e Bug 28803: (follow-up) Error details improvement
This patch updates the exceptions thrown by Koha::Email to include the
parameter that failed email validation and then updates the failure code
to include this parameter and finally display this field in the template.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 10:22:36 +02:00
Jonathan Druart 27affe637a Bug 28870: Use Email::Address->parse
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 10:22:36 +02:00
Jonathan Druart ddb24eb403 Bug 28870: Move email address validation to a specific class method
To ease testing and future changes if needed.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 10:22:36 +02:00
Tomas Cohen Arazi 23aac004c0 Bug 28870: Use Email::Address to validate email addresses
This patch adds a new dependency, Email::Address. It is used in
Koha::Email to replace the current use of Email::Valid, which proved to
be problematic when it comes to UTF-8 characters.

Email::Address provides suitable regexes that -when used- keep our
tests passing, but also deal better with UTF-8 data.

To test:
1. Apply the regression tests patch
2. Notice the only change is that it tweaks a couple addresses so they
   contain umlauts and also have the "Description <address>" format that
   is used when sending carts.
3. Run:
   $ kshell
  k$ prove t/Koha/Email.t
=> FAIL: Tests fail! Things die because Email::Valid doesn't like the
   from we passed.
4. Run:
   $ sudo apt install libemail-address-perl
5. Apply this patch
6. Repeat 3
=> SUCCESS: Tests pass!
7. Try what is described in comment 1
=> SUCCESS: Things are back to normal
8. Sign off :-D
9. Send cookies

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 10:22:35 +02:00
Andrew Isherwood df92c9ddf9 Bug 28340: Alphabetise metadata for notices
This commit sorts metadata by key prior to preparing a notice with it

Test plan:

- Do not apply patch
- Create a report with the following SQL:
  SELECT * FROM message_queue;
- Create an ILL request with plenty of metadata
- Go to "Manage request"
- Send a notice to a patron
- Run the report created earlier

=> TEST:
- Observe that the metadata is in random order

- Apply the patch
- Create an ILL request with plenty of metadata
- Go to "Manage request"
- Send a notice to a patron
- Run the report created earlier

=> TEST:
- Observe that the metadata is now sorted alphabetically by key

Signed-off-by: Jill Tivey <Jill.Tivey@sasa.gov.scot>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
Andrew Isherwood 3fc80c35aa Bug 22614: Migrate original request
This patch changes an active request's backend and initiates the
migration on that request, rather than creating another and migrating
that.

The majority of the work to accomplish this bug is being carried out in
the backends, since Koha delegates migrations to them.

Signed-off-by: Barry Cannon <bc@interleaf.ie>
Signed-off-by: Assumpta Byrne <abyrne@ait.ie>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:57 +02:00
Tomas Cohen Arazi 5e497a47a2 Bug 29032: Pre-load ILL backends to speed up response
Bug 22440 will rewrite the route and make it even more efficient by
prefetching the related data instead of performing several queries in
loops.

In the meantime, we can make this controller perform better with a
simple intervention: load backends once, and use the
$request->_backend() setter to pre-set it before using the objects.

To test:
1. Perform any usual ILL requests listing, try having several
=> FAIL: Notice it takes a weird amount of time to load
2. Apply this patch
3. Restart all
4. Repeat 1
=> SUCCESS: It feels fast enough!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-21 20:22:56 +02:00
Jonathan Druart d587341d34 Bug 17600: Remove wrong C4::Context imports
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:44:27 +02:00
Jonathan Druart e176c4edce Bug 28972: DBIC schema changes
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Martin Renvoize 5bbafce720 Bug 28316: (QA follow-up) Make clean_search_term public
With all the work that's gone into improving the internal
_clean_search_term method I feel we should expose it publically as it's
going to be more widely helpful

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Petro Vashchuk cc988b2d16 Bug 28316: escape ES ranges if QueryAutoTruncate is enabled
if QueryAutoTruncate enabled we will have any special operators ruined
for example: "test [6 TO 7]" will be converted to "test* [6* TO* 7]"
so no reason to keep ranges when QueryAutoTruncate set to "enabled"

1) enable QueryAutoTruncate at your sysprefs.
2) perform a search using range, for example: "[1999 TO 2020]",
it shouldn't work the way it's supposed to.
3) apply the patch.
4) perform the same search with range, ensure that it works correctly.

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Petro Vashchuk 3ce2810324 Bug 28316: avoid messing up regexes in the search queries
This patch ensures that the behavior with
QueryRegexEscapeOptions set to values other than
"Escape" still will works as expected.

It does so by storing the contents of regexes
before escaping special characters and
then restores the contents of regexes back to how
it was before, ensuring that searching with regex is possible.

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Petro Vashchuk 0083f96bc9 Bug 28316: escape exclamation signs in the query
Currently having exclamation sign at the end of the query makes ES
search fail, and when you try to search for a book that has exclamation
sign in the tittle (something like "Words! words") won't show results
correctly as it tries to negate everything that is after exclamation
sign, making it impossible to search for books that have in in the title

This patch escapes exclamation signs if it's at the end of the query or
has a space after it, resolving both of the issues listed above.

To reproduce:
1) with ES enabled, search for the book with title that contains
exclamation sight at the end, like "book!", this search should result
 in error.
2) do another search, but this time find/prepare beforehand book with a
title that has exclamation sign with a space after it,
e.g "exclamation! sign", it shouldn't find it as ES treats everything
after that exclamation sign as negation.
2) apply the patch.
3) perform searches from the steep one and two again.
Search from step one should no longer fail, while search from the step
two should find that book.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Petro Vashchuk fa39f9a65e Bug 28316: escape brackets in the search query
This patch screens square and curly brackets which have no special
language meaning.

To reproduce:
1) using ES, search for the book with title that contains
square and/or curly brackets, like "book [second edition]", which will
result in error.
2) apply the patch.
3) search for that book again, ensure that it works now.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Petro Vashchuk 999f8ba0a8 Bug 28316: screen unquoted semicolons and all followup colons
Currently searches like: "book:", ":book" and "host-item:test:n"
cause internal server errors.

This patch adds additional regexes that remove the colons at the start
and end of the query, and another regex that screens all follow-up
colons that go after the first colon to avoid errors when searching for
"host-item:test:n".

To reproduce:
1) using ES, search for the book with title that contains
semicolon at the start or at the end of the line, separated with spaces,
this should cause internal server error.
2) try doing the same with something like "host-item:test:n", it should
result in error as well.
3) apply the patch.
4) repeat steps 1-2, ensure that it works now.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Nick Clemens 777c519c5a Bug 28847: Move SearchLimitLibrary code to QueryBuilder Modules
This patch removes the code from the search scripts into QueryBuilder
modules.

To test:
1 - Have a library group defined as a search group for both staff and opac
2 - Search on staff client and opac with that group limit and a single branch limit
3 - Note your results/counts
4 - Note the visuals of the search description
5 - Apply patch
6 - Repeat searches
7 - All should work as before

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Kyle M Hall 3b678b6383 Bug 28228: Supress plugin related error message
Use of uninitialized value in subroutine entry at /usr/share/koha/lib/Koha/Plugins/Base.pm line 182.

Test Plan:
1) Install the Kitchen Sink plugin ( it does not have a max versio defined )
2) Note the warning in your logs
3) Apply this patch
4) Restart all the things
5) No warning!

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 14:04:48 +02:00
Martin Renvoize 51f978696e Bug 28960: Explicitly call get_column
This patch adds get_column to ensure we are returning a string value of
a field rather than accidentally triggering a relationship accessor.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-20 12:06:53 +02:00
Tomas Cohen Arazi a4d285a9c4 Bug 26195: Add a way to specify authorised values should be expanded [OAI]
This patch introduces a new configuration entry for OAI so the resulting
records include authorised value descriptions instead of codes.

This is off by default.

To test:
1. Create a yaml file with the extended OAI configuration
2. Visit http://kohadev.myDNSname.org:8080/cgi-bin/koha/oai.pl
3. Browse some records
=> SUCCESS: The fields tied to AV show codes.
4. Update your configuration with 'expanded_avs: 1' for one  of the
   defined format like in:

  ---
  format:
    marc21:
      metadataPrefix: marc21
      metadataNamespace: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim
      schema: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd
      include_items: 0
    marcxml:
      metadataPrefix: marcxml
      metadataNamespace: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim
      schema: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd
      include_items: 1
      expanded_avs: 1
5. restart_all
6. Repead 3
=> SUCCESS: AV descriptions are returned!
7. Sign off :-D

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
Kyle M Hall 57e95681d4 Bug 28306: Fix POD
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
Julian Maurice 4605304f62 Bug 28306: Allow to query database with minimal memory footprint
The goal is to be able to build a database handler (dbh) and to execute
queries without loading unnecessary stuff. This will be useful to reduce
memory usage of daemons that need to check the database
periodically

The patch provides a new method Koha::Database::dbh which returns a
database handler without loading the DBIx::Class schema. This method is
also used by DBIx::Class, so whether you use DBI or DBIx::Class, the
same method is used to initialize the connection.

The patch also moves some code in order to avoid loading C4::Context:
- C4::Context::timezone moves to Koha::Config
- C4::Context::db_scheme2dbi moves to Koha::Database

To measure memory usage I used the following commands:

* before the patch:
perl -MKoha::Database \
    -E 'Koha::Database->schema->storage->dbh->do("select 1");' \
    -E '$|=1; say $$; sleep 2' \
    | while read pid; do ps -p $pid -o rss=; done

* after the patch:
perl -MKoha::Database \
    -E 'Koha::Database->dbh->do("select 1");' \
    -E '$|=1; say $$; sleep 2' \
    | while read pid; do ps -p $pid -o rss=; done

It will give you the RSS (Resident Set Size) of the perl process in kB

What I get:
* before the patch: between 96.9MB and 97.2MB
* after the patch: between 17.8MB and 18.2MB

Note that if a timezone is configured (either from $KOHA_CONF or
TZ environment variable), Koha will load DateTime::Timezone to check if
it's valid, and it increases RSS to 36MB

Another interesting metric is the number of modules loaded:
* before the patch:
perl -MKoha::Database \
    -E 'Koha::Database->schema->storage->dbh;' \
    -E 'say scalar keys %INC'

Result: 567

* after the patch:
perl -MKoha::Database \
    -E 'Koha::Database->dbh;' \
    -E 'say scalar keys %INC'

Result: 51

Test plan:
1. Apply the patch & restart starman
2. Make sure Koha is still ok (ie. can access the database, does not
have encoding issues, ...)
3. Run the tests in t/Context.t, t/Koha/Config.t,
t/db_dependent/Koha/Database.t, t/timezones.t

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
Nick Clemens 70d2deb103 Bug 28774: Don't store blank values for rental discount
This patch adds 'can_be_blank => 0' for the rentaldiscount rule to prevent
storing blank values in the database

Additionally, if there is no charge we do not need to check for a discount
and can simply return

To test:
1 - Set rental discount to "" to a rule in circulation rules
2 - Checkout an item that will follow this rule
3 - Check the intranet log:
    [WARN] Argument "" isn't numeric in subtraction (-) at /kohadevbox/koha/C4/Circulation.pm line 3385.
4 - Apply patch and restart all
5 - Update database
6 - Set the rule to "" again
7 - Check the DB, no rule is stored
    SELECT * FROM circulation_rules WHERE rule_name = 'rentaldiscount';
8 - Checkout the item again
9 - No warns in log

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-31 10:27:52 +02:00
Petro Vashchuk 2b6a79be4c Bug 28736: fix requirements for the correct error message to show up
There is existing error message in the code stating:
"Unable to understand your search query, please rephrase and try again."
which fits perfectly but because it looks for "ParseException" in the
warning output it doesn't show up on this page as it's actually
"parse_exception".

This patch makes that it's also checked if "parse_exception" is present
in the warning output.

Side note:
"ParseException" reaction code was added here:
    e0f6c4dc Bug 12478: improve error reporting a bit
Search::Elasticsearch seems propagates clean ES JSON answer,
and in current ES version inside of $@ it contains "parse_exception"
string in dumped JSON answer ("'type' => 'parse_exception'").
Old seeked phrase "ParseException" wasn't reproduced, only in ES logs
("Caused by: org.apache.lucene.queryparser.classic.ParseException:
 Cannot parse ..."). Check for both phrases won't complicate future
changes, but this note added for reference and code cleanup if needed.

To reproduce:
1) using ES search for something like "// ^ ! { } [ ] .. , <>" that
will for sure break the syntax of ES.
2) after the search query fails note that the error is
"Unable to perform your search. Please try again."
3) apply the patch
4) search for the same thing again
5) error message should be "Unable to understand your search query,
please rephrase and try again." now.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Andrew Nugged <nugged@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:02:07 +02:00
Martin Renvoize deece7b76b Bug 28830: Add cni index for 003
This patch adds the cni/Control-number-identifier index to enable
searches to use the 003 field.

Test plan
1/ Apply patch
2/ Re-index using updated configurations
3/ Confirm cni:number searches yield the expected results
4/ Signoff

Split-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pasi Kallinen <pasi.kallinen@koha-suomi.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:02:07 +02:00
Tomas Cohen Arazi d41b197ff6 Bug 28744: Better handling of undefined to_api_mapping
We always had some mapping because of the DB vs. API object definition discrepancies. But bug 25260 revamps the reserves table, and it is a perfect match. It highlights this edge case: if no mapping defined, then and undef from_api_mapping is generated (as opposed to an empty hashref) and this leads to errors in the query translation from the API.

This patch makes a small change so this method always returns an empty
hashref.

To test:
1. Apply the regression tests
2. Run:
   $ kshell
  k$ prove t/db_dependent/Koha/Object.t
=> FAIL: Test fail!
3. Apply this patch
4. Repeat 2
=> SUCCESS: Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

JD amended patch: remove uneeded commented lines

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:02:07 +02:00
Fridolin Somers 7baea6631a Bug 28554: In Koha::AuthorisedValues sort by description
In itemsearch form, there are several filters build with authorized values.
There values should be sorted by description.

Test plan :
1) Create several values and descriptions in authorized values LOC
2) Go to itemsearch
3) See filter by location sorts on description and not on value

Seems change in search_by_marc_field can not be tested in interface

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-30 17:02:07 +02:00
Jonathan Druart d2606a1c83 Bug 22690: Add koha_object[s]_class to fix TestBuilder.t
'Can't locate object method "_new_from_dbic" via package "Koha::Linktracker" (perhaps you forgot to load "Koha::Linktracker"?) at /kohadevbox/koha/Koha/Object.pm line 334.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 17:06:09 +02:00
Jonathan Druart 46a1342226 Bug 22690: Remove uneeded return and add no_triggers
* C4/Items.pm
  - Koha::Biblios not used

* Koha/Item.pm
  - Koha::Item->orders must return an empty set if no order attached
  - no_triggers should be passed to other update calls

* Item.t
  - No need to build a fund
  - Add new test to test Koha::Item->orders when no order attached

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
Jonathan Druart d21f626e80 Bug 22690: DBIC schema changes
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 15:28:23 +02:00
Martin Renvoize 020b93bfc0 Bug 22690: (QA follow-up) Fix indexing for Items sets
This patch adds tests and handling for calling move_to_biblio on a
Koha::Items set that contains items from more than one source biblio.

Test plan
1/ Inspect the changes to t/db_dependent/Koha/SearchEngine/Indexer.t
2/ Run t/db_dependent/Koha/SearchEngine/Indexer.t and confirm it passes

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:41 +02:00
Joonas Kylmälä 3d5349b914 Bug 22690: (QA follow-up) Correct variable name
The $from_biblio variable name doesn't exists after a refactoring that
happened. Here we need to re-index both the $self biblio and $to_biblio
biblio.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:41 +02:00
Martin Renvoize c958dec785 Bug 22690: (QA follow-up) Add TrackedLink classes and use them
This patch adds Koha::TrackedLink(s) classes based on Koha::Object(s)
and then adds the relationship accessor to Koha::Item and uses it within
the move_to_biblio method.

Tests for new relationship also added to t/db_dependent/Koha/Item.t

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:41 +02:00
Martin Renvoize 2e5bfbc5a7 Bug 22690: (QA follow-up) Use relationship accessor
With the addition of foreign key relationships to the linktracker table
we now get a DBIC relationship accessor we can use. This clarifies the
code slightly by using the _result->relationship form to get the DBIC
resultset.  We should still introduce a Koha::Object based class for
this table at some point.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:41 +02:00
Martin Renvoize e552b36686 Bug 22690: (QA follow-up) Clarify uses of DBIC
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:41 +02:00
Martin Renvoize 2ab8f50835 Bug 22690: (QA follow-up) Move adopt_items_from_biblios to Koha::Items
This patch moves the Koha::Biblio->adopt_items_from_biblio method to the
Koha::Items set class and updates all calls from
Biblio2->adopt_items_from_biblio(Biblio1) to Biblio->items->move_to_biblio(Biblio2)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:41 +02:00
Martin Renvoize 33dc5da155 Bug 22690: (QA follow-up) Improve negation syntax
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:41 +02:00
Martin Renvoize 56b86cf35a Bug 22690: (QA follow-up) Rename 'item_orders' to 'orders'
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:41 +02:00
Joonas Kylmälä 2ca69b49af Bug 22690: (QA follow-up) Index also source biblio when calling move_to_biblio()
We need to update the search index record for the old biblio where the
item was moved from to keep the item info in search index up-to-date.

To test:
1) $ prove t/db_dependent/Koha/SearchEngine/Indexer.t

Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:41 +02:00
Ere Maijala 2615ee1010 Bug 22690: Refactor merging of records to improve performance (Elasticsearch)
This patch allows merging of records with many items without the web server timing out.

Test plan:

Without the patch:

- Create 2 records (one with e.g. 1000 items).
- Do a cataloguing search that displays both records, select them and click "Merge selected".
- Choose the record with many items as the one to be eliminated.
- Start the merging.
- After a while the web server should give you a timeout error (the merging process may still continue)

With the patch:
- Do the same as above
- This time verify that the records are merged without timeout
- Create a new biblio with an item
- Add with the item:
  * acquisition order
  * hold (reserve)
- Merge the biblio to another one
- Verify that the item and its related data was moved
- Verify that tests pass:
  prove -v t/db_dependent/Koha/Biblio.t
  prove -v t/db_dependent/Koha/Item.t
  prove -v t/db_dependent/Koha/SearchEngine/Indexer.t

Signed-off-by: Michal Denar <black23@gmail.com>
Rebased-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-27 11:02:40 +02:00
Jonathan Druart 85c3954606 Bug 24387: Fix Letters/TemplateToolkit.t
We must cast to date or the comparison does not work

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 15:18:10 +02:00
Jonathan Druart 266aacc1f9 Bug 24387: DBIC schema changes
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:55 +02:00
Jonathan Druart 89ade834d6 Bug 24387: Rename "News" with "Additional contents"
One big patch for one big move.

The "News" feature (opac_news) has been hijacked to handle some system
preferences (bug 26050). The goal was to take profit of the UI (editor)
and the ability to translate the value.

Disclaimer: This patch is NOT offering the best implementation but, as
we still don't have bug 24975, it cannot be done now. And no, we don't
want to wait for it to move forward here. This patch is going into the
right direction anyway.

This enhancement is going to rename the "News" with a more genertic
"Additional contents". We have two different "categories" of content:
"news" and "html customizations".

What does it bring?
- A split on the UI for disambigate the two types of content (news and
syspref/html customizations)
- A simplification of the edit form: all languages will be translatable
on the same view (like the "notice templates")
- Ground will be prepared for different types of content (if needed later)
- Staff news can be translated

How was the "News" area working before this patch?
The opac_news DB table contained a (very inconsistent) 'lang' column.
The different values were:
- '' => news to display at the OPAC and staff interfaces
- 'koha' => news for staff only
- 'slip' => news for slip notices
- $lang => news for OPAC only, translated in $lang ('en', 'es-ES', etc.)
- "$location_$lang" => A syspref moved to this "news" area. The syspref
is $location, and is translated in $lang. Eg. OpacLoginInstructions_en,
OpacLoginInstructions_fr-FR, opacheader_es-ES

This patch is improving the DB structure with the following changes:
- renaming 'opac_news' with 'additional_contents'
- new 'category' column
 => 'news' or 'html_customizations'
- new 'location' column
 => For 'news': 'staff_and_opac', 'staff_only', 'slip'
 => For 'html_customizations': the old syspref name (eg. 'OpacLoginInstructions').
- new 'code' column (see later for more info)
- the 'lang' column will only contain the language code ('en', 'es-ES',
etc.). BUT a 'default' entry will ALWAYS exist for fallback behaviour.

We are getting closer to the 'notice template' table structure because
we want to match its UI. The 'code' column will bring us the ability to
group the different 'additional_contents' rows. The code for a given
news will be the same, but the (lang, title, content) will differ.
Examples:
  News 1 will have, for each of the translated versions
  (category, code,    location, branchcode)
  ('news',   'News1', $location, $branchcode||undef)
  And the 3 following columns will differ:
  (title,              content,               lang)
  ('title for news 1', 'content for news 1', 'default')
  ('titulo para 1',    'contenido para 1',   'es-ES')

Note that the "category" is not strictely necessary, but it seems better
to have the ability to split the different content by category/type
easily.

Additional changes:
- Syspref 'NewsToolEditor' is renamed 'AdditionalContentsEditor'
- Koha::NewItem => Koha::AdditionalContent
- Koha::News => Koha::AdditionalContents
- Script and template renamed from koha-news to additional-contents
- Foreign keys have been renamed
- Subpermission edit_news has been renamed edit_additional_contents
- The UI can now be accessed via a "News" or "HTML customizations" link
from the tools module. The related contents will then be displayed (both
categories are now split)

Changes not done here:
- Primary key 'idnew' could be renamed 'id'

Limitations of the upgrade:
News cannot be grouped by a unique code for existing translations.
=> A given news will be now displayed several times on the translated
interface
Any ideas to improve the upgrade behaviour?
We will have to add a warning in the release notes to tell libraries to
review their news.

Test plan:
0. Don't apply the patches
1. Translate the interfaces in some languages
. Create some news for staff and OPAC
. Create some content for different entry of HTML customizations
Note that you are forced to define a 'default'.
Also note that you are only forced to fill the title (not the content).
This is certainly problematic (see FIXME in the code) as sometime only
the content is displayed.
. Play with the interface (edit, delete, filter)
. Go to the different places the news are displayed, and confirm they
are displayed correctly (staff home, opac home, opac rss)
. Create 1+ news for 'slip', check an item out and 'print slip' (from
the circulation page). You must see the news.
. Go to the different places you are expecting the HTML customizations
to be present and confirm that you see them.
. Switch the lang of the interface and confirm that you now see the
content in the translated version
. Generate the templates in another language, don't translate the
content
. Use this language for the interface and confirm that the 'default'
version is displauyed.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-16 11:55:55 +02:00
Martin Renvoize c090f98ff3 Bug 28813: DBIC Schema Update
Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-11 12:53:54 +02:00
Martin Renvoize 2ba7010899 Bug 22435: (follow-up) Set CREATE offset for credits to positive
The history view of an accountline shows increases and decreases in the
amount outstanding from the perspective of the credit/debit line you are
inspecting.

Apply creates a negative offset amount which acts to 'reduce' both the
outstanding 'debt' for the debit line and the outstanding 'credit' for
the credit line.  Starting with a negative offset for the credit thus
makes this nonsenical.

Void creates positive offset amounts which act to 'restore' both the
outstanding 'debt' for the debit line and the outstanding 'credit' for
the credit line.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
Martin Renvoize 57e4176f64 Bug 22435: Update DBIC Schema's
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
Martin Renvoize 37c795d589 Bug 22435: (follow-up) Update POS for reduced offset types
This patch updates the Koha::Charges::Sales class to use the 'CREATE'
and 'APPLY' offset types.

Test plan
1/ Run t/db_dependent/Koha/Charges/Sales.t

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
Martin Renvoize c475d4d97b Bug 22435: Update accountline-details page to show all history
The accountline-details page took a simplistic approach to displaying
the history of an accountline. This patch drops the now defunct
offset_type_description template block from it's respective include
files, and then updates the accountline-details template and controller
to show the full history of what's happend to the accountline passed.
This includes the creation, any increments/decrements (for fines), and
finally any offsets against the total (payments, cancellations, voids).

Test plan
1/ Create some credits and debits and apply them to each other in
various combinations.
2/ View the 'Details' page for some of the debit/credits
3/ Compare before patch and after for the Details page.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
Martin Renvoize dec9f754f8 Bug 22435: Fix _set_found_trigger
The _set_found_trigger utilised the 'Writeoff' offset type to
distinguish between the application of a writeoff and any other form of
offset application.

This patch updates the trigger to use the full link through from offset
to account credit line to get the credit type being offset.

Test plan
1/ Run t/db_dependent/Koha/Items.t and prove it fails before the patch,
but passes after applying this patch.
2/ Run t/db_dependent/Koha/Account/Offsets.t and prove it passes both
before and after applying this patch.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
Martin Renvoize c188ec1567 Bug 22435: Fix CREATE offset amounts
Updateing the tests highlighted a mistake in the original code.. the
offsets for the cancel and reduce creation lines were incorrectly
signed.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00
Martin Renvoize b5a09e4654 Bug 22435: ->apply() should always use 'APPLY' for offset_type
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-08-04 14:06:43 +02:00