This patch adds a new background job for pseduonymization
To test:
1 - Download the benchmarking script
2 - Ensure your koha has bcrypt_settings in koha-conf.xml
See bug 28911
3 - perl perf_check_pseudo.pl
4 - Note the slowdown after pseudonymization enabled
5 - Apply patches, restart all
6 - perl perf_check_pseudo.pl
7 - Note improvement
8 - Enable pseudonymization in sytem preferences
9 - Perform some checkouts and returns
10 - Verify the background jobs complete successfully
11 - Verify the pseudonymized_transactions table is updated correctly
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Whne performing batch operations we can send a large numebr of records for reindexing at once.
Currently this can create requetss that are too large for Elasticsearch to process. We need
to break these requests into chunks/
This patch adds a chunk_size configuration to the elasticsearch stanza in koha-conf.xml
If blank we default to 5000.
To test:
0 - Have Koha using Elasticsearch
1 - Create and download a report of all barcodes:
SELECT barcode FROM items
2 - Batch modify these items
3 - Note a single ESindexing job is created
4 - Create and download a report of all authority ids:
SELECT auth_header.authid FROM auth_header
5 - Setup a marc modification template, and batch modify all the authorities
6 - Again note a single ES backgorund job is created
7 - Apply patch
8 - Repeat the modifications above - you still get a single job
9 - Edit koha-conf.xml and add <chunk_size>250</chunk_size> to elasticsearch stanza
10 - Repeat modifications - you now get several background ES jobs
11 - prove -v t/db_dependent/Koha/SearchEngine/Elasticsearch/Indexer.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test plan:
[1] Try public endpoint like /api/v1/public/biblios/X/items.
* Replace X by an existing biblionumber.
You should receive a 200.
[2] Enable OPACMaintenance. Set to Show.
[3] Try same endpoint. You should get a 503 with a json error.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch addresses the fact some of our code still relies on those CGI-era ENV variables.
In particular, C4::Laguages::getlanguage expects it to calculate the most suitable language.
As this is reported as required, I provide a fix.
I did a `git grep HTTP_` on the codebase, and the only other occurences are:
* C4::Output (HTTP_X_REQUESTED_WITH)
* C4::Context (HTTP_USER_AGENT, HTTP_X_FORWARDED_FOR)
* C4::Auth_with_shibboleth (using HTTP_ variables for matchpoints)
* Some handling on the 'Koha as a Mojo app' code
* Some .pl script not relevant to this report
I decided to limit this patch to the requested header which seems harmless. I leave the
other cases.
To test:
1. Apply the regression tests patch
2. Run:
$ ktd --shell
k$ qa -c 1
=> FAIL: Tests fail!
3. Apply this patch
4. Run:
k$ qa -c 2
=> SUCCESS: Tests pass! All green!
5. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Koha::Plugins::get_enabled_plugins is called by Koha::Plugins::call
which is called by C4::Templates->new, which is used in the web
installer when the database is not created yet
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a "get_descriptions_by_marc_field" method
which caches AuthorisedValue descriptions when searched by
MARC field, which is used when exporting MARC to CSV.
Test plan:
0. Do not apply the patch yet!
1. Go to http://localhost:8081/cgi-bin/koha/tools/csv-profiles.pl
2. Add a CSV profile with the default values and using the following for the "Profile MARC fields":
000|001|003|005|006|007|008|010|015|016|020|022|040|050|082|100|110|111|130|240|243|245|246|247|250|260|300|336|337|338|440|490|500|501|505|520|530|600|610|611|630|648|650|651|690|700|710|711|856|887|942|995|952|952$c
3. Create a new List
4. Add all the database's bibs to that list using SQL like the following (where the shelfnumber equals the number for your list):
insert into virtualshelfcontents (shelfnumber,biblionumber,borrowernumber) select 1,biblionumber,51 from biblio;
5. Go to that list in the staff interface
6. Download that list using your CSV profile
7. Apply the patch
8. koha-plack --reload kohadev
9. Download that list using your CSV profile
10. Note that the output is the same, but that the output completes much more quickly after applying the patch
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds some missing links from the preservation module to the manual.
To test:
1. Go to Preservation > Trains
2. Click on a train name
--> You are brought to the Show train section of the Preservation chapter of the manual
3. Click Edit next to an item
--> You are brought to the Add items section of the Preservation chapter of the manual (there is no section specific to editing items, but the form to edit is the same as the form to add)
4. Go back to the trains list, and click the Edit button next to one of the trains
--> You are brought to the Trains section of the Preservation chapter of the manual (there is no section specific to editing trains yet)
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch links the various pages of the preservation module to each
specific section of the preservation module chapter in the manual.
To test:
1. Enable the PreservationModule system preference
2. Go to the Preservation module
3. Click "help" in the top right corner
--> You are brought to the manual, but not a specific page
4. Apply patch
5. Click "Help" in the top right corner
--> You are brought to the Preservation chapter of the manual
6. Click "Settings"
7. Click "Help" in the top right corner
--> You are brought to the Settings section of the Preservation chapter of the manual
8. Choose a not for loan status for items added to the waiting list (the mandatory field) and submit
9. Click "Add a new processing"
10. Click "Help" in the top right corner
--> You are brought to the Add a new processing section of the Preservation chapter of the manual
11. Fill out the form to create the processing and submit
12. Click "Edit this processing"
13. Click "Help" in the top right corner
--> You are brought to the Add a new processing section of the Preservation chapter of the manual (there is no edit a processing section)
14. Click "Waiting list"
15. Click "Help" in the top right corner
--> You are brought to the Waiting list section of the Preservation chapter of the manual
16. Add items to the waiting list
17. Click "Trains"
18. Click "Help" in the top right corner
--> You are brought to the Trains section of the Preservation chapter of the manual
19. Click "New train"
20. Click "Help" in the top right corner
--> You are brought to the New train section of the Preservation chapter of the manual
21. Fill out the form to create a new train and submit
22. Click "Add items"
23. Click "Help" in the top right corner
--> You are brought to the Add items section of the Preservation chapter of the manual
24. Also check ERM help links to make sure nothing is broken
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a new method to the Koha::Notice::Message class to wrap
content in the same way we currently do at email send time. We then
update the notice preview to use this method so we're embedding
NoticeCSS into that iframe. This means the generated preview should now
exactly match how your notice will display in email clients including
any CSS you may have placed within NoticeCSS.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Fixed typo in POD of html_content
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Currently when a patron with an expired password is authenticated via the API a 500 error is returned rather than a 400 "Validation failed" error. This patch catches the return value for an expired password and returns the validation failure before the patron search is attempted.
Test plan:
1) Choose a patron and set their password expiry date to a date in the past
2) Send a request to auth/password/validation as an authenticated user with that patron's details
3) The response should be a 500 error
4) Apply patch
5) Repeat steps 1-3 and this time the response should be a 400 code with an error message of "Password expired"
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Yet another approach ;)
Driemaal is scheepsrecht / Third time lucky..
Test plan:
Run prove $(git grep -l Koha::Plugin | grep -P "^t\/db")
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
No longer falls in an endless loop if the last month in the report period is december.
Test plan:
1) Apply just the tests patch, run EUsage tests
prove t/db_dependent/Koha/ERM/EUsage/
2) Wait for a few seconds. Notice SushiCounter.t it hangs. CTRL+C (cancel it)
3) Apply the fix patch.
4) Run the tests again. Notice all EUsage tests pass with no issues.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Auld <andrew.auld@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Spelling.
Tidyness.
Removal of leftover warn
Nick's patch fixes the issue as described, and I agree that it is the ideal solution here.
prove t/db_dependent/api/v1/erm_eholdings* passes
prove t/db_dependent/Koha/BackgroundJob/CreateEHoldingsFromBiblios.t passes
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
These patches move the store routine from Koha::ERM::EHoldings::Resource to Koha::ERM::EHoldings::Title as the code deals exclusively with title fields.
It updates the code to ensure that records are created when a title is not attched to a biblio, and that only the biblio title field is updated when updating an eholdings title.
To test:
1 - Enable ERMModule sys pref
2 - Create a new public list, visit:
/cgi-bin/koha/virtualshelves/shelves.pl
3 - Click "New list" enter name, set public -> public
4 - Click "Add items", enter 112\n113\n114 (new line for each), in "Biblio numbers"
5 - Notice that all added biblios have quite a few MARC fields
6 - Go to packages, visit:
/cgi-bin/koha/erm/eholdings/local/packages
7 - Create a new package, add a name and hit 'Submit'
8 - Go to titles, visit:
/cgi-bin/koha/erm/eholdings/local/titles
9 - Click "import from list"
10 - Pick the package created in 7)
11 - On the row of the list created in 2), click "Import"
12 - Go back to the list, visit:
/cgi-bin/koha/virtualshelves/shelves.pl?op=view&shelfnumber=1
13 - Notice all the biblios have been stripped of their MARC data, and only title remains.
14 - Go to details page for one of the bibs
15 - Edit -> Replace record via Z39.50
16 - You can search for anything, just make sure to import a mostly full record
17 - Go back to ERM - eHoldings - Local - Titles
18 - Edit the title for the record that was replaced
19 - Save
20 - Go to record details - note the record is gone again
21 - Apply patches
22 - Search the catalog
23 - Check some titles
24 - Add to a new list
25 - repeat 8 - 11 with the new list
26 - View and confirm records are intact
27 - Edit the eholdings title for one of the records, changing the title field
28 - Save and view record details
29 - Confirm record is intact and title updated
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
We should not just look at Click.
Test plan:
[1] Apply the example patch for a Blur action in barcode.pl.
[2] Check that barcode.pl is active in your framework and that
autoBarcode is not disabled.
[3] Open the browser dev console.
[4] Tab a few times through barcode in the neworderempty form of
Acquisition. If you leave the field, the Blur event fires.
=> Without this patch you will see two console lines (FF may
show a little '2' icon at the right side) added. With this
patch just one.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jan Kissig <jkissig@th-wildau.de>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
test plan:
a) build a new basket + order line using "item create on placing an order"
b) in the item creation dialog (neworderempty.pl) please use the webbrowsers dev tools to analyze the [...] button next to subfield d (Date acquired) and inspect the events that get fired by clicking
c) notice the Clicktag_952_subfield_d_* event gets fired twice.
d) apply patch and reload page
e) there should be only 1 click handler left (besides the jquery one), which points to the Clicktag_952_subfield_* function
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
See also bug 35298. You may have seen that the Focus event of dateaccessioned
did not fire. But that is related to use of Flatpickr in this plugin.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Add two requirements when registering a new patron:
- A child patron must have a guarantor. This is controlled by
a new syspref ChildNeedsGuarantor.
- A guarantor cannot be a guarantee.
Test plan:
1. Add a child patron without guarantor or child patron with guarantee
as guarantor succesfully.
2. Apply this patch.
3. Add a child patron as a guarantor.
=> Error is raised.
4. Turn syspref "ChildNeedsGuarantor" ON.
5. Add a child patron without a guarantor and error "Child needs a
guarantor" is raised.
6. Add guarantor. Guarantor can either be existing patron or added with
"Contact" section.
=> Save without errors.
Also prove t/db_dependent/Koha/Patron.t
Sponsored-by: Koha-Suomi Oy
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Added to the call in opac-password-recovery.pl. This allows
differentiating between password change and password reset when
viewing the logs.
Test plan:
Enable BorrowersLog.
Do a password recovery on OPAC.
Check with log viewer for 'RESET PASS' action.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kelly McElligott <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds a new module to letters 'patron_slip' / Patrons toolbar
(slip)
This also combines some of the slip printing JS to make things more
direct
To test:
1 - Apply patch
2 - restart_all
3 - yarn build
4 - Confirm you can print slip, summary, quickslip, and checkin slip
when viewing a patron's account
5 - Enable DisplayClearScreenButton syspref, and test that it works
with both values
6 - Go to Tools->Notices and slips
7 - Define a new notice in the 'Patron toolbar (slip)' module
8 - Enter content like below in the 'Print' transport:
Patron has [% borrower.holds.count %] holds
List of holds:
[% FOREACH hold IN borrower.holds %]
===============================
Reserve id: [% hold.reserve_id %]
Title [% hold.biblio.title %]
[% END %]
9 - Go to patron scree
10 - Note new option under 'Print'
11 - Print the new slip and confirm it works (try with and without
placing holds for patron
Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch moves code form scripts to a template plguin and unifies the calls
To test:
1 - Add a searchable patron attribute type
2 - Add values to several patrons and test patron searching from
Patrons home
Patrons search results
Holds request screen
Article request screen
Patron search sidebar
3 - Apply patch
4 - Confirm results are the same
Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch renames all occurences of the method.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
* Perltidies to pass the QA script
* Adds missing say statement to the database update
* Makes sure the tests pass on a database, where UpdateItemLocaton*
system preferences are not empty
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patchset shifts the logic for changing the item location on checkin
(controlled by the UpdateItemLocationOnCheckin system preference) to a
new subroutine in Koha/Items.pm
That subroutine logic is shared with the UpdateItemLocationOnCheckout
system preference.
Test plan:
1. Apply patches, update databases and restart services
2. Set the following system preferences:
- UpdateItemLocationOnCheckin:
FIC: PROC
- UpdateItemLocationOnCheckout:
PROC: FIC
3. Checkout an an item with items.location = 'PROC'. Observe it's
location is changed to 'FIC'
4. Return the item. Observe it's location is changed to 'PROC'
5. Change UpdateItemLocationOnCheckout to:
PROC: _BLANK_
6. Issue the item with items.location = 'PROC' and confirm it's location
is blanked on checkout
7. Issue and return an item with a different location e.g. 'REF' (don't
use 'CART' as this is blanked by bug 14576 on checkout).
Observe the location does not change on issue or return.
8. Run unit tests
sudo koha-shell kohadev
prove t/db_dependent/Circulation/issue.t
Sponsored-by: Toi Ohomai Institute of Technology, New Zealand
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This change adds the "protected" column to patron import.
Test plan:
0. Apply the patch and run "restart_all"
1. prove -v t/db_dependent/Koha/Patrons/Import.t
2. Go to http://localhost:8081/cgi-bin/koha/tools/import_borrowers.pl
3. Note "protected" is listed in the "fields you want to supply"
on the right side of the screen
4. Note "protected" appears in file downloaded by clicking
"Download a Starter CSV file with all the columns"
5. Note that "Protected" appears after clicking "Enter default values"
and after clicking "Preserve existing values"
6. For bonus marks, use the "Starter CSV file" to import a test patron
(remember surname, branchcode, and categorycode are the required fields)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds a check for biblio existance prior to method resolution
failure in Koha::Template::Plugin::Biblio.
This fixes a failure case in unit tests.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch replaces the uses of the static method with the new filter.
It also moves the tests to the relevant place and removes the (now)
unused method.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds a RecordProcessor filter class that takes care of
performing the field cleanup that was originally done in the static
method `Koha::MetadataRecord::stripWhitespaceChars`.
In order to test the results, I adapted the tests to use the filter
instead of the original method.
To test:
1. Run:
$ ktd --shell
k$ prove t/Koha_MetadataRecord.t
=> SUCCESS: Tests pass
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests still pass
4. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The updated code removed the pushing of fields to an array - this caused a bug.
When multiple fields were copied and replaced and there were more fields added than existed originally we were adding the new field to the end, then removing the first occurence of the original field. If we tried to move 2 650 fields to 651 and the record had no 651 we would:
- Delete the first 651, there were none, so nothing happened
- Take the first 650, add it to the end of the 651 group (there were none, so it became the first 651)
- Delete the first 651, which was the field we just copied
- Take the second 650 and add it to the end of the 651 group (whihc had none, because we deleted it)
I re-add the line, but do as suggesed by Phil and reverse the order (unshift vs push)
To test:
1 - Apply other patches
2 - prove -v t/SimpleMARC.t
3 - It fails
4 - Apply this patch
5 - prove -v t/SimpleMARC.t
6 - It still fails, but more tests pass (there's another patch to follow)
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This updates the SimpleMARC tests and MarcModificationTemplates
tests to expect the new order of fields
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Test plan :
- create a marc modification template which add a new field to a record
- apply that template to some records
- verify that the new field is always appended at the bottom of the record, even if some fields are higher.
- apply patch
- apply template to another set of records
- verify the added field is now ordered within the already existing fields in the records
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Kyle M Hall
Marcel de Rooy
Nick Clemens
Tomas Cohen Arazi
Julian Maurice
David Cook
Katrin Fischer
Caroline Cyr La Rose
Martin Renvoize
Matt Blenkinsop
Pedro Amorim
Jonathan Druart
Jan Kissig
Emmi Takkinen
Katrin Fischer
Alex Buckley
Arthur Suzuki