Issue description:
- call to ModZebra was unconditional inside 'store' method for Koha::Item,
so it was after each item added, or deleted.
- ModZebra called with param biblionumber, so it is the same parameter
across calls for each items with same biblionumber, especially when we
adding/removing in a batch.
- with ElasticSearch enabled this makes even more significant load
and it is also progressively grows when more items already in DB
Solution:
- to add extra parameter 'skip_modzebra_update' and propagate it down to
'store' method call to prevent call of ModZebra,
- but to call ModZebra once after the whole batch loop in the upper layer
Test plan / how to replicate:
- make sure that you have in the admin settings "SearchEngine" set to
"Elasticsearch" and your ES is configured and working
( /cgi-bin/koha/admin/preferences.pl?op=search&searchfield=SearchEngine )
- select one of biblioitems without items
( /cgi-bin/koha/cataloguing/additem.pl?biblionumber=XXX )
- press button "add multiple copies of this item",
- enter 200 items, start measuring time and submit the page/form...
On my test machine when adding 200 items 3 times in a row (so 600 in
total, but to show that time grows with every next batch gradually):
WHEN ElasticSearch DISABLED (only Zebra queue):
- 9s, 12s, 13s
WHEN ElasticSearch ENABLED:
- 1.3m, 3.2m, 4.8m
WITH PATCH WHEN ElasticSearch ENABLED:
- 10s, 13s, 15s
Same slowness (because also same call to ModZebra) happens when you try
to delete all items ("op=delallitems"). And same fix.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes as_marc_field skip subfields that don't have a value or
contains an empty string (replicating C4::Items:1021 logic).
To test:
1. Apply the regression tests patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/Item.t
=> FAIL: Tests fail because the generated MARC::Field contains undef
subfields.
3. Apply this patch
4. Repeat 2.
=> SUCCESS: Tests pass! Undefined and empty subfields are skipped!
5. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This is the only situation I found where:
* t/db_dependent/Koha/Item.t is passing
* t/db_dependent/Koha/Object.t is passing
* MySQL 8 is happy (and not fail with "Column 'timestamp' cannot be null"), which is certainly what I missed in the previous follow-up
About the change to Object.t: the next store was called without the
generated timestamp, so it is needed to call discard_changes.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Owen Leonard 2018-03-16 10:47:47 UTC :
<<
I don't think the system preference adds any security. There are already multiple permissions required for working with plugins:
- Configure plugins
- Manage plugins ( install / uninstall )
- Use report plugins
- Use tool plugins
And even with those permissions your server must be configured to allow the use of plugins.
>>
Test plan :
1) Install kitchen sink plugin https://github.com/bywatersolutions/koha-plugin-kitchen-sink
2) Run misc/devel/install_plugins.pl
3) Set config enable_plugins=1
4) Check all parts of the plugin are working
5) Set config enable_plugins=0
6) Check all parts of the plugin are disabled
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We had a unique behvaiour where the syspref was set to string 'NULL'
as opposed to undef, we need to clean that up
To test:
1 - Set OpacRenewalBranch to 'NULL' in staff interface
2 - Renew via opac
3 - Check statistics to ensure branch is blank
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patchset moves all code to calculate the correct renewal branch into Koha::Item.pm
When interface is opac we follow the syspref, otherwise we use the current userenv, or pass through
a defined branch
To test:
1 - Check out an item to a patron
2 - Set allowed renewals in the circ rules to 100 (just so you can keep testing)
3 - Renew the item in staff interface, confirm it is recorded correctly in statistics table (as signed in branch)
4 - Renew via the opac, testing with each setting of OpacRenewalbranch
5 - prove -v t/db_dependent/Koha/Item.t
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
get_dirty_columns only work for existing items.
This fixes t/db_dependent/ShelfBrowser.t
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
For discussion, this patch revert the changes made previously.
This line exists in Koha::Item->store as it is the translation of:
if (exists $item->{'timestamp'}) {
delete $item->{'timestamp'};
}
that was coming from _do_column_fixes_for_mod (called from ModItem)
To preserve existing behavior I would be in favor of keeping it like that to
avoid regression, and deal with it separately if we want to improve/remove it.
So basically here we are setting it to undef in Koha::Item->store to make it
handle correctly by the parent Koha::Object->store. I agree that's kind of weird
and must be improved.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This sounds wrong as we should let the DBMS do that, but it was failing.
Here we are doing the same as Koha::Patron->store for dateenrolled
To recreate the failure, prove t/db_dependent/Koha/Item.t without this
patch:
DBD::mysql::st execute failed: Column 'timestamp' cannot be null [for
Statement "UPDATE `items` SET `more_subfields_xml` = ?, `timestamp` = ?
WHERE ( `itemnumber` = ? )" with ParamValues: 0='<?xml version="1.0"
encoding="UTF-8"?>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Few occurrences have been added since this patchset has been originaly
written
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
No need to calculate cn_sort if not dirty when we store
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This change to Koha::Item->store seems correct here, but tests from /db_dependent/Items.t is failing now.
Adjusting them to make them pass.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We do not want to log if nothing changed
How could we do that better?
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
From C4::Items to Koha::Item
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Starting to replace the ModItem calls with Koha::Item->store
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We are done with AddItem, only need to log and index.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Remove the unique call to _set_derived_columns_for_add
Even if design to deal with other values it only dealts with cn_sort
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
_set_defaults_for_add is no longer needed as it is handled by dbic and koha::object->store
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 9978 should have fixed them all, but some were missing.
We want all the license statements part of Koha to be identical, and
using the GPLv3 statement.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes the following methods return array references of
Koha::Library objects instead or unblessed objects;
- Koha::Item->pickup_locations
- Koha::Biblio->pickup_locations
- Koha::Libraries->pickup_locations
Bonus:
- The template plugin is adjusted to unbless things to keep behavior
- Tests are moved to the right .t file.
- Tests for the new behavior are added.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds "patron's hold group" as a new option to Hold pickup library match
To test:
1. Set ReservesControlBranch preference to item.
2. Create a hold group
3. Go to circulation and fines rules
SUCCESS => in 'Hold pickup library match' there is a new option called "patron's hold group"
4. In a library not in hold group set 'Hold policy' to "any" and 'Hold pickup library match' to "patron's hold group"
5. Search for a user in the hold group
6. 'Search to hold' for items of the library of step 4
7. Select an item and 'Place hold for [user]'
SUCCESS => in 'Pickup at' you should see patron's hold group as options
8. In OPAC sign in as the same user of step 5
9. Search for the item in step 7
SUCCESS => in 'Pick up locations' you should see patron's hold group as options
10. Sign off
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
* Bug 22284: (follow-up) Remove commented warn and address test failures
* Bug 22284: (follow-up) fix test count after merge
* Bug 22284: (follow-up) fixes after 15496
* Bug 22284: (follow-up) fixes after 18936
* Bug 22284: (follow-up) Remove HomeOrHolding from reserves
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch modifies Koha::Libraries->pickup_location and moves most of the logic to
Koha::Item and Koha::Biblio in preparation for api endpoints in the future.
There where 2 methods added
1) Koha::Item->pickup_locations that given a patron, returns all pickup locations of
this item, considering hold fulfillment rules, and hold allowed rules.
2) Koha::Biblio->pickup_locations that given a patron, returns a distinct list of
libraries returned by each of this biblio items pickup location.
Koha::Libraries->pickup_location analyzes input param and calls Koha::Item->pickup_locations
or Koha::Biblio->pickup_locations as needed.
Also in opac-reserve.tt the way options where obtained to fill the pickup location select
was modified to pass the patron as a parameter.
To test:
1) opac: try to place hold on a item and check that all libraries are shown in the
pickup location select.
2) intranet: in Library groups, add 2 root groups marked as local hold group and
add different libraries to each.
3) opac: login as a user of a library belonging to one hold group, and search try to
place a hold on an item belongin to the other hold group.
4) intranet: in Circulation and fines rules, play with 'Hold policy' and 'Hold pickup
library match' rules.
5) opac: On each modification of the rules reload the page.
SUCCESS => Every time you reload the page, the number of pickup locations showed in
select varies.
6) prove t/db_dependent/Koha/Biblios.t t/db_dependent/Koha/Items.t
SUCCESS => Result: PASS
7) Sign off
Sponsored-by: VOKAL
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
The system preference AllowItemsOnHoldCheckout only affects SIP based checkouts. I believe we should rename it AllowItemsOnHoldCheckoutSIP so it matches AllowItemsOnHoldCheckoutSCO. We should also update the description as well.
As far as I know AllowItemsOnHoldCheckout has never affected anything except SIP traffic. That does not preclude the addition of another AllowItemsOnHoldCheckout syspref that would affect the staff interface, but that would be an enhancement and is outside the scope of this bug report.
Test Plan:
1) Test AllowItemsOnHoldCheckout via SIP
2) Apply this patch
3) Test AllowItemsOnHoldCheckoutSIP via SIP
4) Note there is no difference!
5) Note the syspref description has been update to mention SIP
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds a to_api_mapping method to the class. This in effect
enables calling ->to_api on the object. The mapping is borrowed from the
API controller. It is not removed from the controller so we are able to
verify (through the tests) that there is no behavior change.
Once this is pushed we need to implement the counter-wise methods and
clean the controllers.
To test:
1. Run:
$ kshell
k$ prove t/db_dependent/api/v1/items.t
=> SUCCESS: Tests pass
2. Apply this patch
3. Repeat (1)
=> SUCCESS: Tests still pass!
4. Sign off :-D
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jan Kolator <kolatorj@email.cz>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Changed the POD of the new method to use holds.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch changes pending_hold to has_pending_hold to signify that
we're returning a boolean and not a Koha::Hold object.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
To test:
1 - Enable course reserves
2 - Create a course
3 - Add an item to the course
4 - Attempt to view the course on the OPAC
5 - Internal Server Error
6 - Apply patch
7 - Add an item to the holds queue by placing a hold and running holds queue builder or:
INSERT INTO tmp_holdsqueue (itemnumber) VALUES (###);
8 - View the course page, note item appears 'Pending hold'
9 - Remove the holdsqueue line
10 - View the course page, note item appears 'Available'
11 - prove -v t/db_dependent/Koha/Item.t
Signed-off-by: Hayley Mapley <hayleymapley@catalyst.net.nz>
I see Jonathan's comments about small improvements, but will sign off as
everything works as expected here.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch makes Koha::Item->hidden_in_opac take hidelostitems into
account when performing the calculation.
To test:
- Apply the regression tests patch
- Run:
$ kshell
k$ prove t/db_dependent/Koha/Item.t
=> FAIL: Tests fail because the code doesn't (yet) care about the
syspref
- Apply this patch
- Run:
k$ prove t/db_dependent/Koha/Item.t
=> SUCCESS: Tests pass!
- Sign off :-D
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch adds a hidden_in_opac method that does the same calculation
done in GetHiddenItemnumbers, but for a single item, and doesn't get the
OpacHiddenItems syspref, but expects them to be passed as parameters (to
avoid multiple reads).
To test:
- Apply this patches
- Run:
$ kshell
k$ prove t/db_dependent/Koha/Item.t
=> SUCCESS: Tests pass!
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
The stock rotation feature adds a batch process to automate rotation of
catalgue items with a staff client page under tools to manage rotas/schedules.
Once a rota is configured, and your staff user has the right permissions
to allocate items, then an additional tab will appear on biblio records
allowing the management of of which rota, if any, individual items belong to.
It also includes a cron script to process the items on a daily basis.
Signed-off-by: Kathleen Milne <kathleen.milne@cne-siar.gov.uk>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Edit: I removed a temporary file
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
I do not think we should allowed branchode and Koha::Library objects, it
adds confusion in callers (we never know if we have a branchcode of a
library object).
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch adds a new method Koha::Biblio->can_be_transferred. The method checks
if at least one of the item of that biblio can be transferred to desired location.
This method will be useful for building a smarter pickup location list for holds,
because we will be able to hide those libraries to which none of the items of
this biblio can be transferred to due to branch transfer limits (see Bug 7614).
To test:
1. prove t/db_dependent/Koha/Biblios.t
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch changes Koha::Item->can_be_transferred to accept a HASHref as follows:
$item->can_be_transferred({ to => $library, from => $library2 })
To test:
1. prove t/db_dependent/Koha/Items.t
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Jonathan Druart
Andrew Nugged
Tomas Cohen Arazi
Fridolin Somers
Nick Clemens
Kyle M Hall
Agustin Moyano
Katrin Fischer
Martin Renvoize
Lari Taskula