Commit Graph

5022 Commits

Author SHA1 Message Date
Matt Blenkinsop df9d4b0f55 Bug 35204: Prevent an expired password from throwing a 500 error
Currently when a patron with an expired password is authenticated via the API a 500 error is returned rather than a 400 "Validation failed" error. This patch catches the return value for an expired password and returns the validation failure before the patron search is attempted.

Test plan:
1) Choose a patron and set their password expiry date to a date in the past
2) Send a request to auth/password/validation as an authenticated user with that patron's details
3) The response should be a 500 error
4) Apply patch
5) Repeat steps 1-3 and this time the response should be a 400 code with an error message of  "Password expired"

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit f3bb885052)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2024-01-03 15:51:13 +01:00
Marcel de Rooy 4fb9fad791 Bug 35507: Add verbose parameters in Koha::Plugins
Yet another approach ;)
Driemaal is scheepsrecht / Third time lucky..

Test plan:
Run prove $(git grep -l Koha::Plugin | grep -P "^t\/db")

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
(cherry picked from commit d59bf322e9)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2024-01-03 15:15:24 +01:00
Tomas Cohen Arazi df84f4a5e7
Bug 34587: DBIC schema
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-30 14:58:49 -03:00
Pedro Amorim 9988925fb5
Bug 35418: Fix _get_usage_months
No longer falls in an endless loop if the last month in the report period is december.

Test plan:
1) Apply just the tests patch, run EUsage tests
prove t/db_dependent/Koha/ERM/EUsage/
2) Wait for a few seconds. Notice SushiCounter.t it hangs. CTRL+C (cancel it)
3) Apply the fix patch.
4) Run the tests again. Notice all EUsage tests pass with no issues.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Auld <andrew.auld@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-29 17:36:11 -03:00
Pedro Amorim 55f024c0e3
Bug 35115: (QA follow-up):
Spelling.
Tidyness.
Removal of leftover warn

Nick's patch fixes the issue as described, and I agree that it is the ideal solution here.
prove t/db_dependent/api/v1/erm_eholdings* passes
prove t/db_dependent/Koha/BackgroundJob/CreateEHoldingsFromBiblios.t passes

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-29 17:36:10 -03:00
Nick Clemens f984969690
Bug 35115: Move store routine from Resource to Title and update code
These patches move the store routine from Koha::ERM::EHoldings::Resource to Koha::ERM::EHoldings::Title as the code deals exclusively with title fields.

It updates the code to ensure that records are created when a title is not attched to a biblio, and that only the biblio title field is updated when updating an eholdings title.

To test:
 1 - Enable ERMModule sys pref
 2 - Create a new public list, visit:
     /cgi-bin/koha/virtualshelves/shelves.pl
 3 - Click "New list" enter name, set public -> public
 4 - Click "Add items", enter 112\n113\n114 (new line for each), in "Biblio numbers"
 5 - Notice that all added biblios have quite a few MARC fields
 6 - Go to packages, visit:
     /cgi-bin/koha/erm/eholdings/local/packages
 7 - Create a new package, add a name and hit 'Submit'
 8 - Go to titles, visit:
     /cgi-bin/koha/erm/eholdings/local/titles
 9 - Click "import from list"
10 - Pick the package created in 7)
11 - On the row of the list created in 2), click "Import"
12 - Go back to the list, visit:
     /cgi-bin/koha/virtualshelves/shelves.pl?op=view&shelfnumber=1
13 - Notice all the biblios have been stripped of their MARC data, and only title remains.
14 - Go to details page for one of the bibs
15 - Edit -> Replace record via Z39.50
16 - You can search for anything, just make sure to import a mostly full record
17 - Go back to ERM - eHoldings - Local - Titles
18 - Edit the title for the record that was replaced
19 - Save
20 - Go to record details - note the record is gone again
21 - Apply patches
22 - Search the catalog
23 - Check some titles
24 - Add to a new list
25 - repeat 8 - 11 with the new list
26 - View and confirm records are intact
27 - Edit the eholdings title for one of the records, changing the title field
28 - Save and view record details
29 - Confirm record is intact and title updated

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-29 17:36:09 -03:00
Jonathan Druart 588e834e7d Bug 35174: Do not explode if po dir does not exist
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2023-11-16 08:56:36 +01:00
Marcel de Rooy e9326f0e86
Bug 35012: Apply change to all events
We should not just look at Click.

Test plan:
[1] Apply the example patch for a Blur action in barcode.pl.
[2] Check that barcode.pl is active in your framework and that
    autoBarcode is not disabled.
[3] Open the browser dev console.
[4] Tab a few times through barcode in the neworderempty form of
    Acquisition. If you leave the field, the Blur event fires.
    => Without this patch you will see two console lines (FF may
    show a little '2' icon at the right side) added. With this
    patch just one.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jan Kissig <jkissig@th-wildau.de>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-14 10:32:40 -03:00
Jan Kissig ec3fe5d41e
Bug 35012: Prevent Click handler firing twice
test plan:
a) build a new basket + order line using "item create on placing an order"
b) in the item creation dialog (neworderempty.pl) please use the webbrowsers dev tools to analyze the [...] button next to subfield d (Date acquired) and inspect the events that get fired by clicking
c) notice the Clicktag_952_subfield_d_* event gets fired twice.
d) apply patch and reload page
e) there should be only 1 click handler left (besides the jquery one), which points to the Clicktag_952_subfield_* function

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
See also bug 35298. You may have seen that the Focus event of dateaccessioned
did not fire. But that is related to use of Flatpickr in this plugin.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-14 10:32:39 -03:00
Kyle M Hall b0c4bc3623
Bug 12133: (QA follow-up) Tidy for qa script
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-10 10:59:42 -03:00
Emmi Takkinen d9a989eaf0
Bug 12133: Add requirements for guarantor and guarantee
Add two requirements when registering a new patron:

- A child patron must have a guarantor. This is controlled by
  a new syspref ChildNeedsGuarantor.
- A guarantor cannot be a guarantee.

Test plan:
1. Add a child patron without guarantor or child patron with guarantee
as guarantor succesfully.
2. Apply this patch.
3. Add a child patron as a guarantor.
=> Error is raised.
4. Turn syspref "ChildNeedsGuarantor" ON.
5. Add a child patron without a guarantor and error "Child needs a
guarantor" is raised.
6. Add guarantor. Guarantor can either be existing patron or added with
"Contact" section.
=> Save without errors.

Also prove t/db_dependent/Koha/Patron.t

Sponsored-by: Koha-Suomi Oy
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-10 10:59:40 -03:00
Marcel de Rooy a08dd73621
Bug 21431: Add action parameter to set_password
Added to the call in opac-password-recovery.pl. This allows
differentiating between password change and password reset when
viewing the logs.

Test plan:
Enable BorrowersLog.
Do a password recovery on OPAC.
Check with log viewer for 'RESET PASS' action.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-09 14:50:11 -03:00
Tomas Cohen Arazi beecec2b59
Bug 34517: (QA follow-up) Standardize boolean attributes definition
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-09 10:15:08 -03:00
Tomas Cohen Arazi 83612d5ecb
Bug 34517: DBIC schema
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-09 10:15:08 -03:00
Nick Clemens 72b106d0b5
Bug 34517: Add boolean to search columns for schema
Signed-off-by: Kelly McElligott <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-09 10:15:06 -03:00
Nick Clemens 48494c357c
Bug 32986: (follow-up) Fix CircAutoPrintQuickSlip and remove unecessary module usage
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 17:59:35 -03:00
Nick Clemens 9d2567a6f8
Bug 32986: (follow-up) POD
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 17:59:35 -03:00
Nick Clemens c3b40a5499
Bug 32986: Add patron_slip module to letters and allow printing on members account
This patch adds a new module to letters 'patron_slip' / Patrons toolbar
(slip)

This also combines some of the slip printing JS to make things more
direct

To test:
1 - Apply patch
2 - restart_all
3 - yarn build
4 - Confirm you can print slip, summary, quickslip, and checkin slip
  when viewing a patron's account
5 - Enable  DisplayClearScreenButton syspref, and test that it works
  with both values
6 - Go to Tools->Notices and slips
7 - Define a new notice in the 'Patron toolbar (slip)' module
8 - Enter content like below in the 'Print' transport:
Patron has [% borrower.holds.count %] holds

List of holds:
[% FOREACH hold IN borrower.holds %]
===============================
    Reserve id: [% hold.reserve_id %]
    Title [% hold.biblio.title %]
[% END %]
9 - Go to patron scree
10 - Note new option under 'Print'
11 - Print the new slip and confirm it works (try with and without
  placing holds for patron

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 17:59:33 -03:00
Nick Clemens cf70fd27da
Bug 34519: (follow-up) fix QA tool complaints
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 17:52:39 -03:00
Nick Clemens 9d69ae986a
Bug 34519: Add a template plugin for fetch searchable patron attributes
This patch moves code form scripts to a template plguin and unifies the calls

To test:
1 - Add a searchable patron attribute type
2 - Add values to several patrons and test patron searching from
    Patrons home
    Patrons search results
    Holds request screen
    Article request screen
    Patron search sidebar
3 - Apply patch
4 - Confirm results are the same

Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 17:52:37 -03:00
Tomas Cohen Arazi 353d239ab7
Bug 34438: DBIC schema
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 11:41:16 -03:00
Tomas Cohen Arazi 554f791491
Bug 35269: Improve POD
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 09:58:56 -03:00
Tomas Cohen Arazi 97b1e6a77d
Bug 35269: Rename `update_item_location` to `location_update_trigger`
This patch renames all occurences of the method.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-08 09:58:55 -03:00
Katrin Fischer f534e799ed
Bug 21159: (QA follow-up) Add say to database update, perltidy, fix tests
* Perltidies to pass the QA script
* Adds missing say statement to the database update
* Makes sure the tests pass on a database, where UpdateItemLocaton*
  system preferences are not empty

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-06 11:34:49 -03:00
Alex Buckley 81826a7af7
Bug 21159: Update item location on checkout
This patchset shifts the logic for changing the item location on checkin
(controlled by the UpdateItemLocationOnCheckin system preference) to a
new subroutine in Koha/Items.pm

That subroutine logic is shared with the UpdateItemLocationOnCheckout
system preference.

Test plan:
1. Apply patches, update databases and restart services

2. Set the following system preferences:
- UpdateItemLocationOnCheckin:
FIC: PROC

- UpdateItemLocationOnCheckout:
PROC: FIC

3. Checkout an an item with items.location = 'PROC'. Observe it's
location is changed to 'FIC'

4. Return the item. Observe it's location is changed to 'PROC'

5. Change UpdateItemLocationOnCheckout to:
PROC: _BLANK_

6. Issue the item with items.location = 'PROC' and confirm it's location
is blanked on checkout

7. Issue and return an item with a different location e.g. 'REF' (don't
use 'CART' as this is blanked by bug 14576 on checkout).
Observe the location does not change on issue or return.

8. Run unit tests
sudo koha-shell kohadev
prove t/db_dependent/Circulation/issue.t

Sponsored-by: Toi Ohomai Institute of Technology, New Zealand
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-06 11:34:48 -03:00
David Cook 5414d696bc
Bug 35264: Add "protected" column to patron import
This change adds the "protected" column to patron import.

Test plan:
0. Apply the patch and run "restart_all"
1. prove -v t/db_dependent/Koha/Patrons/Import.t
2. Go to http://localhost:8081/cgi-bin/koha/tools/import_borrowers.pl
3. Note "protected" is listed in the "fields you want to supply"
on the right side of the screen
4. Note "protected" appears in file downloaded by clicking
"Download a Starter CSV file with all the columns"
5. Note that "Protected" appears after clicking "Enter default values"
and after clicking "Preserve existing values"

6. For bonus marks, use the "Starter CSV file" to import a test patron
(remember surname, branchcode, and categorycode are the required fields)

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-06 08:42:50 -03:00
Martin Renvoize 470da48af8
Bug 29002: (follow-up) Make CanBook more resilient
This patch adds a check for biblio existance prior to method resolution
failure in Koha::Template::Plugin::Biblio.

This fixes a failure case in unit tests.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 14:22:49 -03:00
Tomas Cohen Arazi cfc63f77bb
Bug 35190: DBIC schema
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 14:22:48 -03:00
Kyle M Hall ef9d9a006a
Bug 33749: (QA follow-up) Tidy code for qa script
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:08:07 -03:00
Tomas Cohen Arazi b6d39225bc
Bug 33749: Use TrimFields instead of stripWhitespaceChars
This patch replaces the uses of the static method with the new filter.
It also moves the tests to the relevant place and removes the (now)
unused method.

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:08:06 -03:00
Tomas Cohen Arazi 54477ecc51
Bug 33749: Add TrimFields field class
This patch adds a RecordProcessor filter class that takes care of
performing the field cleanup that was originally done in the static
method `Koha::MetadataRecord::stripWhitespaceChars`.

In order to test the results, I adapted the tests to use the filter
instead of the original method.

To test:
1. Run:
   $ ktd --shell
  k$ prove t/Koha_MetadataRecord.t
=> SUCCESS: Tests pass
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests still pass
4. Sign off :-D

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:08:06 -03:00
Nick Clemens 498f0dfecc
Bug 24480: (follow-up) Shift new fields into array and add after all are copied
The updated code removed the pushing of fields to an array - this caused a bug.
When multiple fields were copied and replaced and there were more fields added than existed originally we were adding the new field to the end, then removing the first occurence of the original field. If we tried to move 2 650 fields to 651 and the record had no 651 we would:
- Delete the first 651, there were none, so nothing happened
- Take the first 650, add it to the end of the 651 group (there were none, so it became the first 651)
- Delete the first 651, which was the field we just copied
- Take the second 650 and add it to the end of the 651 group (whihc had none, because we deleted it)

I re-add the line, but do as suggesed by Phil and reverse the order (unshift vs push)

To test:
1 - Apply other patches
2 - prove -v t/SimpleMARC.t
3 - It fails
4 - Apply this patch
5 - prove -v t/SimpleMARC.t
6 - It still fails, but more tests pass (there's another patch to follow)

Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:34 -03:00
Arthur Suzuki 48fd8d1900
Bug 24480: Unit test
This updates the SimpleMARC tests and MarcModificationTemplates
tests to expect the new order of fields

Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:34 -03:00
Arthur Suzuki d3112dc319
Bug 24480: Fix insert_fields_ordered instead of append_fields in SimpleMARC
Test plan :
 - create a marc modification template which add a new field to a record
 - apply that template to some records
 - verify that the new field is always appended at the bottom of the record, even if some fields are higher.
 - apply patch
 - apply template to another set of records
 - verify the added field is now ordered within the already existing fields in the records

Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:33 -03:00
Tomas Cohen Arazi 579887db9e
Bug 29002: DBIC schema
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:32 -03:00
Tomas Cohen Arazi 5ed3758b97
Bug 29002: (QA follow-up) Correct param retrieval in controllers
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:31 -03:00
Tomas Cohen Arazi 6b7e493448
Bug 29002: Tidy
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:30 -03:00
Martin Renvoize 3b1612cf27
Bug 29002: Circulation
This patch adds handling for circulation of booked materials.

We prevent checkouts of booked items during the booking period where the
booking is for another user.

We update the due date to match the end of booking period when checking
out to the user who the booking is for.

If a checkout would overlap a booking to another user, we ask the user if
they will accept a reduced due date so the booked item is returned in
time to fulfill the booking.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:26 -03:00
Martin Renvoize 131a87567b
Bug 29002: Prevent booking a checked out item
This patch updates the bookings logic to prevent placing a booking
on an item that is checked out to start prior to it's due date.

Test plan
1) Follow the previous test plans
2) Add a checkout of an item
3) Attempt to place a booking and note that the item is not available to
   be booked until after it's due date.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:25 -03:00
Martin Renvoize 2c6d57c254
Bug 29002: Assign item at time of booking
This patch updates the logic to ensure we always assign an item at the
point of booking instead of assuming we can assign the item at fulfilment time.

This makes the logic significantly simpler on the client when trying to
calculate available dates.

Test plan
1) Attempt to create a new booking using the 'Any item' option in the
   modal
2) Note that upon placing your booking you are assigned an item for that
   booking.

NOTE: We add a FIXME to the client code to highlight something to test
when/if someone decides to have another try and moving the item
assigment to just in time as aposed to up front.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:19 -03:00
Martin Renvoize 6bfe8467cb
Bug 29002: (follow-up) Add bookings count to side menu
This patch adds booking counts to the bookings option in the side menu
for biblio display pages.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:14 -03:00
Martin Renvoize 1a7d3efcab
Bug 29002: Schema update
DBIx::Class schema update for new 'bookable' field in the items table
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:12 -03:00
Martin Renvoize 2472214301
Bug 29002: Add bookable state to items
This patch adds a bookable boolean to enable/disable the ability to book
an item ahead of time

Test plan
1) Navigate to the 'Items' tab of a biblio
2) Note the new 'Bookable' option and select at least one item to allow
   bookings to take place
3) Note that without any items selected as 'bookable' one does not have
   the 'Place booking' option or the 'Bookings' tab on the biblio
   details page.
4) Note that when at least one item is bookable, the place booking modal
   now only displays items that are marked as bookable in the item
   selection
5) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:11 -03:00
Martin Renvoize 9fff2e1f4e
Bug 29002: Add ability to book material from the staff client
This patch introduces a new modal to the biblio details page to allow
booking of materials.

Test plan
1) Navigate to the details page of a biblio
2) Note the new 'Place booking' button in the toolbar
3) Click the new button and note the new modal dialogue
4) Enter part of a patron name or cardnumber and then select from the
   presented results
5) Optionally pick an item from the select list
6) Select a start date and end date from the calender
7) Submit
8) Attempt to book the same item to another user, note that the dates
   previously selected are now greyed out.
9) Experiment with different items and all items options to confirm the
   available slots in the datepicker update as expected.
10) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:07 -03:00
Martin Renvoize e8d5219547
Bug 29002: Add bookings objects and API classes
This patch adds new Koha::Object based classes for bookings logic and
adds API controllers to expose the new bookings data via the REST API's.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:07 -03:00
Martin Renvoize 1a679de4a1
Bug 29002: DBIC Schema Build
Rebuild of DBIx::Class schema files for the new table and relations
added.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:06 -03:00
Tomas Cohen Arazi 7456c5a483
Bug 18203: DBIC schema
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 18:01:40 -03:00
Katrin Fischer fff910fc56
Bug 8367: (QA follow-up) Fix QA script
* Add +x to atomic database update file
* Perltidy
* Add spans to rewritten tab label for translatability

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 18:01:36 -03:00
Aleisha Amohia ffde32e0d7
Bug 8367: (follow-up) Fix pickup delay text on OPAC and other bits
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 18:01:31 -03:00
Aleisha Amohia 6c0eb32a08
Bug 8367: Add holds_pickup_period circulation rule
So that pickup delay can have a different value per patron category,
   item type or branch.

To test:

1) Update database, restart services
2) Set ReservesMaxPickUpDelay syspref (if not already set)
3) Edit your circulation rules and set a value under 'Holds pickup
period (day) that is DIFFERENT from ReservesMaxPickUpDelay. Set a few
different numbers for different branches as well.
4) Place a hold on a biblio from the staff client.
5) Check in an item from that biblio and confirm the hold as waiting
6) Confirm the expiration date is calculated using the 'Holds pickup
period' value instead of the ReservesMaxPickUpDelay syspref
7) Revert the waiting status and delete the hold
8) Re-place the hold on the biblio on the OPAC. Notice that when you
change the pick up location, the number of days in the pickup message
below the dropdown changes based on the circ rules.
9) Create a holiday with a date that will overlap with the 'Holds pickup
period'
10) Check in an item from that biblio and confirm the hold as waiting
11) Confirm the expiration date is calculated using the 'Holds pickup
period' value AND considers the special holiday
12) Confirm tests pass t/db_dependent/Holds/WaitingReserves.t
13) Test Talking Tech:
13a) Enable TalkingTechItivaPhoneNotification
13b) Go to Tools -> Notices & slips. Add content to the HOLD phone
(itiva) notice.
13c) In your terminal, run perl
/path/to/koha/misc/cronjobs/thirdparty/TalkingTech_itiva_outbound.pl -o
~/itiva.tmp -w 0 --type=RESERVE

Sponsored-by: Catalyst IT

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 18:01:30 -03:00
Tomas Cohen Arazi 78589dbe6e
Bug 34008: Harmonize attribute names
This patch harmonizes the attribute names with what is used for `items`
and `checkouts` in terms of terminology.

It also adapts the tests so they are less random failure-prone (they had
a fixed value for the item type, which might make things explode if the
chosen value already exists on the DB.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:15 -03:00
Tomas Cohen Arazi 05f0604d60
Bug 34008: (QA follow-up) 'item_type_id' should be used
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:15 -03:00
Andreas Jonsson 82bdf93ab7
Bug 34008: Add REST endpoint for list of itemtypes
Test plan:
* Enable the system preference RESTBasicAuth
* curl -s --request GET http://kohadev-intra.mydnsname.org:8081/api/v1/itemtypes
  should give 401 Unauthorized
* curl -s -u koha:koha --request GET http://kohadev-intra.mydnsname.org:8081/api/v1/itemtypes
  should produce JSON-list of itemtypes
* curl -s -u koha:koha --header "x-koha-embed: translated_descriptions" --request GET http://kohadev-intra.mydnsname.org:8081/api/v1/itemtypes
  should include the field translated_descriptions containing the translated descriptions, if any

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] perltidy -b t/db_dependent/api/v1/itemtypes.t # Resolve bad score of 44
[EDIT] chmod 755 t/db_dependent/api/v1/itemtypes.t
[EDIT] perltidy -b Koha/REST/V1/ItemTypes.pm
Lesson: Please run qa tools yourself and adjust accordingly?
Edit (tcohen): I restored the item_type_translated_description.yaml file
as the entire API was broken because of the lack of it.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:14 -03:00
Martin Renvoize 6f693a2420
Bug 35074: Use patron relationship alias
This patch updates the existing Koha::Account::Line->patron accessor to
use the new aliased 'patron' relation in the schema class

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:11 -03:00
Martin Renvoize 465713bb63
Bug 35074: Add patron relationship alias
This patch adds the 'patron' relationship alias to the Acountline
DBIx::Class schema.

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:11 -03:00
Tomas Cohen Arazi 21504e2ab4
Bug 32730: Add FIXME in POD to highlight return value
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:09 -03:00
Michael Hafen 0f122a9400
Bug 32730: Add Patron Lists tab to patron details and circulation pages
Test Plan:
1. Apply patch
2. create a patron list if there aren't any
3. search for a patron
4. observe the "Patron lists" tab showing the list that the patron is not in
5. try adding the patron to the list and removing them from the list to
   be sure the feature has full operation
6. click "Check Out" on the side bar menu to navigate to the circulation
   page for this patron
7. observe the "Patron lists" tab, and verify it operates as it did on the
   patron details page

Bug 32730: (follow-up) Minor corrections
   by Owen Leonard

This patch corrects an instance of an incorrect capital letter
("Patron Lists" -> "Patron lists") and makes minor tweaks to
indentation.

Signed-off-by: Stina Hallin <stina.hallin@ub.lu.se>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Edit: (tcohen) new files should be run through perltidy ALWAYS. Did it
and squashed it here.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:09 -03:00
Tomas Cohen Arazi cf57e2f918
Bug 26170: DBIC booleans
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:08 -03:00
Tomas Cohen Arazi 4317eca9b3
Bug 26170: DBIC schema
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:06 -03:00
Magnus Enger 6fec4ea0df
Bug 26170: Add protected status for patrons
This set of patches makes it possible to protect patrons from being accidetally
deleted or merged with other patrons, from the UI and from (well behaved) cron
jobs. The following subroutines are affected:
- Koha::Patron::delete
- Koha::Patron::merge_with
- Koha::Patron::safe_to_delete
- C4::Members::GetBorrowersToExpunge

Please note:
- This does not intend to protect patrons from being edited, only from being
  deleted

To test:

* Tests
- Run the affected tests:
  prove t/db_dependent/Members.t
  prove t/db_dependent/Koha/Patrons.t

* Editing protected status and manual deletion
- Add a new user, note the presence of the "Protected" field under "Library
  management", but leave it at the default "No", for now.
- Note that "Protected" is displayed in the "Library use" section of the patron
  details.
- Note that More > Delete is avaiable as an action when the patron is saved
- Edit the user and set "Protected" to "Yes"
- Note that More > Delete is now disabled, with a note that the patron is protected

* Batch patron deletion
- Go to Tools > Batch patron deletion and anonymization
- Check the box for "Verify you want to delete patrons"
- Choose the category of your protected patron for "whose patron category is"
  and click "Next" to run the actual deletion
- Check that your protected patron was not deleted

* Merging patrons
- Make sure you have two patrons with similar names or the same category, so
  you can find them with one search. One should be protected, one not.
- Search for the patrons, tick their boxes and click on "Merge selected patrons"
- Select one of the patrons as the "patron to keep".
. Click on "Merge patrons"
- "No valid patrons to merge were found" should be shown
- Repeat this with the other patron as the "patron to keep"
(A future enhancement could be to not allow a protected patron to be selected for
merging in the first place.)

* misc/cronjobs/delete_patrons.pl
- Make sure you have a protected patron, in a category with at least one more
  patron.
- Run something like this (at least in ktd):
  $ perl misc/cronjobs/delete_patrons.pl --category_code <code> -v --confirm
  (Replace <code> with the actual categorycode.)
- Make sure the borrowernumber of the protected patron is not mentioned in the
  output of the script.
- Check the protected patron was not deleted
- Check the non-protected patrons were deleted

* REST API (with ktd)
- Make sure you still have a protected patron, and note their borrowernumber
- Enable RESTBasicAuth and restart all the things
- Run these two commands from the command line on the host:
  $ curl -u koha:koha --request GET "http://localhost:8081/api/v1/patrons/54"
  $ curl -u koha:koha --request DELETE "http://localhost:8081/api/v1/patrons/54"
  (Replace 54 with the actual borrowernumber of your protected patron.)
- The first curl command should give you the patron details. The second should
  give this output:
  {"error":"Protected patrons cannot be deleted","error_code":"is_protected"}

There could be more functions/scripts where patrons are deleted that I have not
thought about. Please report them on the bug if you find any!

Update 2023-10-19: Fix "More > Delete" on patron, so link can not be clicked.
Update 2023-10-19: Rebase

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:06 -03:00
Julian Maurice 8194419144
Bug 33353: Add compatibility with Search::Elasticsearch 8.0
For some reasons, with Search::Elasticsearch 8.0 JSON::true is
interpreted as 1 and Elasticsearch will always respond that it found "at
least 1 record". Something like this:

{
    total => {
        value => 1,
        relation => 'gte'
    }
}

Replacing JSON::true by \1 works with every version of
Search::Elasticsearch I tested (6.80, 7.717, 8.0)

Also worth noting:
Search::Elasticsearch will stop being supported by Elastic
https://github.com/elastic/elasticsearch-perl/issues/220

We might need to find an alternative for future versions of
Elasticsearch

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:04 -03:00
Julian Maurice 24aab6b9c5
Bug 33353: Add compatibility with Elasticsearch 8.x
Elasticsearch 8 removed the support of types. This patch adapts the
requests accordingly.
With this patch, Koha will still be compatible with Elasticsearch 7.x
but will no longer work with Elasticsearch 6.x
Since Elasticsearch 6.x is no longer maintained, this should not be a
problem.

https://www.elastic.co/guide/en/elasticsearch/reference/7.17/removal-of-types.html

Tested with Search::Elasticsearch 6.80 and 7.717

Test plan:
1. Install Elasticsearch 8
   (use docker image: docker.io/koha/elasticsearch-icu:8.x)
2. Change the elasticsearch server location in $KOHA_CONF
3. Run misc/search_tools/rebuild_elasticsearch.pl -r -b -v
   It should fail
4. Apply the patch
5. Run misc/search_tools/rebuild_elasticsearch.pl -r -b -v
   It should end with "Total XXX records indexed"
6. Try to search some biblios and verify that it works the same as
   before

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:04 -03:00
Tomas Cohen Arazi 3f1e21bbca
Bug 35180: DBIC schema
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:03 -03:00
Tomas Cohen Arazi c7571c6a32
Bug 34520: DBIC schema
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:03 -03:00
Nick Clemens 17cf580f19
Bug 34587: (QA follow-up) Tidy
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:32 -03:00
Pedro Amorim a35403cb84
Bug 34587: Add IR_A1 fields to COUNTER report
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:31 -03:00
Matt Blenkinsop 38f8e116fa
Bug 34587: Fix ->validation->output in API endpoints
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:30 -03:00
Pedro Amorim c81112e798
Bug 34587: (QA follow-up): squash
Tidy
$c->validation in API controllers
File permissions
.inc file filters

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:30 -03:00
Pedro Amorim f7e2bc5d9b
Bug 34587: Move YOP to mus/yus instead of erm_usage_titles
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:29 -03:00
Matt Blenkinsop c0b449ea2a
Bug 34587: Add access_type and yop to the reports viewer
Some report types include access_type and yop headers for each usage statistics

The hierarchy is as follows: yop - access_type - metric_type
I.e. for each yop there are report lines for access_type and then for each access_type there are also report lines for metric types

This patch adds helper functions to populate the report rows based on this hierarchy and the report data retrieved from the database

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:28 -03:00
Pedro Amorim ee4db6d763
Bug 34587: Improve title uniqueness rules
Improve robustness of unique matching here, to make sure we always match for same title if the case,

Some report rows may correspond to the same title as the previous row but have an empty match field, or alternatively come with a filled match field (e.g. DOI or Print_ISSN in TR_J4. Because of this we only verify a uniqueness match field if both current row and previous have it non-empty, otherwise we keep checking the remaining uniqueness match fields.

Example of this use-case, COUNTER report:
title    | publisher | platform  | Proprietary_ID | Print_ISSN | DOI     | YOP  | usages
examplet | examplep  | examplepl | 1              | 123        |        | 2020 | usages
examplet | examplep  | examplepl | 1              | 123        |someDOI | 2021 | usages

The above 2 rows is the same title, same publisher, same proprietary_id,
same Print_ISSN, etc. It just so happens that one was returned by SUSHI
with DOI and the other wasnt.
These 2 rows correspond to different usage statistics by YOP, not 2
different usage_titles

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:27 -03:00
Pedro Amorim 184602ce38
Bug 34587: Add proprietary_id and platform to erm_usage_titles
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:26 -03:00
Pedro Amorim 5a08ad2863
Bug 34587: Update database columns length
Some COUNTER files have long data in these fields and were causing new
database entries to fail

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:26 -03:00
Pedro Amorim d55ea648ac
Bug 34587: Improve SUSHI COUNTER error handling
This is doing a few things, checking for sushi errors:
1) Abort background job and provide error message if SUSHI returned
   {Severity} in response, this means SUSHI error, docs here:
https://app.swaggerhub.com/apis/COUNTER/counter-sushi_5_0_api/5.0.2#/SUSHI_error_model

2) Abort background job and provide error message if SUSHI provided
   proper response, but contains Exceptions in Report_Header

3) Abort background job and provide error message if SUSHI returned 0
   report items (e.g. everything is fine but there are no items for the
   harvest period provided

4) Abort background job and provide error message if COUNTER file
   created is larger than allowed by the database.

5) Provide error message if manual upload file size exceeds the max
   allowed by the database

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:25 -03:00
Pedro Amorim e629e37bd7
Bug 34587: Fix total added objects in job report
Remove obsolete total_records code, this was a leftover from when this code was in UsageDataProvider.pm
Add a new job_callback 'added_usage_objects' to handle total objects added (titles, platforms, databases, items)
Update job report include file

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:24 -03:00
Matt Blenkinsop 7ac6b1b72c
Bug 34587: Add report type and provider name to failed background job report
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:23 -03:00
Matt Blenkinsop d4afc38c7c
Bug 34587: Add try catch blocks to object creation
Currently a harvest fails completely if a single title fails to harvest. Rather than failing the entire job, this patch introduces try catch blocks to stop this and allow the job to continue. Failed imports are recorded and displayed in the background job UI on job completion

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:22 -03:00
Matt Blenkinsop 979ff589d9
Bug 34587: Remove counter files from API embedding to improve performance
Rather than fetching the counter files and embedding the counter logs, we now add a foreign key to the data provider in the counter logs table and fetch them directly.

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:21 -03:00
Matt Blenkinsop 040f6ef193
Bug 34587: Update field lengths to 255 characters
(cherry picked from commit 9f384df2ad5f4c037b052e8914d7e4b3e5f26670)
(cherry picked from commit 02c9a57af9f27cf528a2eb924876037890cf5504)

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:21 -03:00
Matt Blenkinsop ebc98070d6
Bug 34587: Fix url validation
We need to check whether the url ends in a trailing "/" and also whether it ends in "/reports" for a SUSHI harvest. Two new functions are added to manage this and prevent failed harvests.

report_type is also now lowercase in the url and the dates are truncated to YYYY-MM format

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:19 -03:00
Pedro Amorim 3ab4036636
Bug 34587: New sushi counter class
Moves a lot of the logic for creating a counter file and parsing a SUSHI response to a new class to simplify logic

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:19 -03:00
Pedro Amorim ca8ea02fda
Bug 34587: Move everything into a ERM/EUsage subfolder
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:18 -03:00
Matt Blenkinsop b7aebb337e
Bug 34587: Add integration with the counter registry API
The counter registry offers an API that provides SUSHI information for each provider. This will be incredibly useful for creating new providers as we can use the correct information from the registry for harvesting urls etc. This reduces the risk of user input errors when creating providers and gives greater reliability to the data required to successfully harvest from the SUSHI API

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:17 -03:00
Matt Blenkinsop 89b6d51f07
Bug 34587: Improve error handling for connection testing
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:16 -03:00
Pedro Amorim cf87082682
Bug 34587: File upload - Background job
ErmSushiHarvester background job will now be initiated by either
enqueue_counter_file_processing_job or by enqueue_sushi_harvest_jobs
from Koha/ERM/UsageDataProvider.pm, the former if triggered by a manual
file upload, the latter if by the 'run now' button or by the cron script.

This commit also includes some rewording/refactoring, namely:
- COUNTER file validation now happens in the API, before enqueuing the job.
- Removal of no longer used POST /erm/counter_files endpoint
- Koha/ERM/UsageDataProvider.pm:
-- run method is now enqueue_sushi_harvest_jobs
-- new enqueue_counter_file_processing_job method
-- harvest method is now harvest_sushi
-- new set_background_job_callbacks method to set the background job callbacks
- REST/V1/ERM/UsageDataProviders
-- run method is now process_SUSHI_response
-- new process_COUNTER_file endpoint

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:15 -03:00
Matt Blenkinsop e5ec3df36e
Bug 34587: Code tidying
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:14 -03:00
Pedro Amorim 9fc2151ead
Bug 34587: Remove begin_date and end_date as usage_data_providers properties/columns
(cherry picked from commit 9339eed9358301f7bf17934e2e13fc17205d9cd0)

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:14 -03:00
Pedro Amorim 16592d267a
Bug 34587: Accept date inputs upon confirming 'run now' button
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:13 -03:00
Pedro Amorim 3318adf48d
Bug 34587: Add a report row for each existing metric_type in a sushi result's performance instance, instead of for reach existing metric_type in the sushi response header, because that might not exist.
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:12 -03:00
Matt Blenkinsop ffa16245d3
Bug 34587: Make start and end dates in providers summary data dependent
Currently the start and end dates in the summary tab are based on the earliest and latest harvest run, rather than the earliest and latest data harvested. This should be changed so that we can see the period of data harvested for each provider

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:12 -03:00
Matt Blenkinsop e935423b78
Bug 34587: Move period total calculations to the backend to improve performance
Remove the front end processing in favour of doing this in the backend and passing it in the API response

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:11 -03:00
Matt Blenkinsop b74815dc64
Bug 34587: Add list endpoints for usage_databases/items/platforms
Needed to display lists in the provider tabs in the UI

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:08 -03:00
Matt Blenkinsop 8987ff1591
Bug 34587: Abstract reports backend to allow new data types
Now that harvesting is possible for platforms, databases and items we need to be able to generate reports for all of these data types. Currently the reporting backend structure is very geared towards titles. Rather than copying this for each different data type, this patch abstracts the code to accept the data type as a url parameter and use that to generate a report based on a given data type

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:07 -03:00
Matt Blenkinsop f57d5088c7
Bug 34587: Add ability to delete saved reports
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:07 -03:00
Pedro Amorim 13356ae74e
Bug 34587: COUNTER file upload
Adds the ability to manually upload a counter file and harvest data to the provider

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:06 -03:00
Matt Blenkinsop e1583a334a
Bug 34587: Add reports by data provider
Add the option to have a report by provider that rolls all usage up into one top-level figure to see how often that provider is being used a given period

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:03 -03:00
Matt Blenkinsop ffbcf83b95
Bug 34587: Add default report capability to reports viewer
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:02 -03:00
Matt Blenkinsop eeb81d9794
Bug 34587: Add publisher and publisher_id to usage_titles schema and KohaTables
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:01 -03:00
Matt Blenkinsop c1455422e4
Bug 34587: Add test connection button
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:45:59 -03:00
Pedro Amorim 2951edc697
Bug 34587: SUSHI COUNTER Harvester
This commit is a squash of the following:

SUSHI harvesting process in the data providers class:
* Builds the URL query and requests the SUSHI service endpoint
* Parses the JSON response and builds the csv COUNTER file and adds it to counter_files table

Usage statistics data processing:
* When a counter_files entry is stored, CounterFile.pm will:
* Parse the csv COUNTER file and
* Add a usage_titles entry for each unique title in the COUNTER file
* Add the title's respective erm_usage_mus (monthly usage) entries, repeating for each metric_type
* Add the title's respective erm_usage_yus (yearly usage) entries, repeating for each metric_type

Harvesting cronjob;

'Run now':
* API endpoint to start the harvesting process of a data provider
* Button in the data providers list to run the harvesting process for each data provider upon clicked

ERM SUSHI: Background job

Job progress is updated to total amount of usage titles after retrieving
the response from SUSHI;
Job warning and success messages are added accordingly
Redundant duplicate titles will not be added
Redundant duplicate monthly and yearly usage statistics will not be added
Data provider harvest background job harvests once per report_type

Enqueue one background job for each report_type in the usage data provider

Update the way we measure progress in the background job.
It now uses the COUNTER report body rows instead of SUSHI response results.
We're now incrementing and showing the number of skipped mus, skipped
yus, added mus and added yus
There's a bug in the way we calculate yus
Updates to background job progress bar - Depends on 34468

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:45:58 -03:00