Two discussions on koha-devel lead to the same conclusion:
biblioitems.marcxml should be moved out this table
- biblio and biblioitems
http://lists.koha-community.org/pipermail/koha-devel/2013-April/039239.html
- biblioitems.marcxml & biblioitems.marc / HUGE performance issue !
http://lists.koha-community.org/pipermail/koha-devel/2016-July/042821.html
There are several goals to do it:
- Performance
As Paul Poulain wrote, a simple query like
SELECT publicationyear, count(publicationyear) FROM biblioitems GROUP BY publicationyear;
takes more than 10min on a DB with more than 1M bibliographic records
but only 3sec (!) on the same DB without the biblioitems.marcxml field
Note that priori to this patch set, the biblioitems.marcxml was not
retrieved systematically, but was, at least, in
C4::Acquisition::GetOrdersByBiblionumber and C4::Acquisition::GetOrders
- Flexibility
Storing the marcxml in a specific table would allow use to store several
kind of metadata (USMARC, MARCXML, MIJ, etc.) and different formats (marcflavour)
- Clean code
It would be a first step toward Koha::MetadataRecord for bibliographic
records (not done in this patch set).
Test plan:
- Update the DBIC Schema
- Add / Edit / Delete / Import / Export bibliographic records
- Add items
- Reindex records using ES
- Confirm that the following scripts still work:
* misc/cronjobs/delete_records_via_leader.pl
* misc/migration_tools/build_oai_sets.pl
- Look at the reading history at the OPAC (opac-readingrecord.pl)
- At the OPAC, click on a tag, you must see the result
Note: Changes in Koha/OAI/Server/ListRecords.pm is planned on bug 15108.
Signed-off-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Zeno Tajoli <z.tajoli@cineca.it>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
There is a TODO in Koha::Patron->delete, this method should call the
Koha::Patron->holds to retrieve the holds made by the patron we are
removing instead of calling Koha::Holds->search
Test plan:
prove t/db_dependent/Koha/Patrons.t
Should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
The goal of this method is to replace
C4::Reserves::GetReservesFromBorrowernumber but could be reused for
something else, that's why it has its own bug report.
Test plan:
prove t/db_dependent/Koha/Patrons.t
should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Rebased. Edited two test descriptions.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This was hardcoded to 0001-01-01.
To test:
- Make sure you have a couple of records, with different timestamps
- Enable OAI-PMH
- Check http://localhost:2201/cgi-bin/koha/oai.pl?verb=Identify
and verify that "Earliest Registered Date" is 0001-01-01
- Apply the patch
- Re-check the "Earliest Registered Date" and verify that it is now
the lowest timestamp in your biblio table
- Manipulate the timestamps and verify that "Earliest Registered Date"
changes accordingly
- Sign off!
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
It would be very useful to be able to define multiple plugin directories
in the Koha conf file. This would allow for ease of plugin development
so that each plugin installed can live in its own git repository. For
compatibility, the first plugindir instance defined should be the one
used for uploading plugins via the web interface.
Test Plan:
1) Apply this patch
2) Define a second pluginsdir line in your koha-conf.xml
3) Clone the kitchen sink plugin to this new path like this:
git clone https://github.com/bywatersolutions/koha-plugin-kitchen-sink.git /path/to/new/plugins/dir
4) Restart memcached if you are running it
5) The Kitchen Sink plugin should now appear in your list of plugins!
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
I rebased it against master and tested it on a kohadevbox
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This is the third patch in a series to unify all payment functions into
a single mathod
Test Plan:
1) Apply this patch
2) prove t/db_dependent/Accounts.t
3) Test fine payment via the "Pay selected" button
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Patron.pm: Adds two missing semicolons at the last statement. Not strictly
needed, but strongly recommended.
Patrons.t: Add a test description, remove two comments that refer to
previously hardcoded dates.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patchset moves the C4::Members::GetUpcomingMembershipExpires
subroutine code to the Koha::Patrons->search_upcoming_membership_expires
method.
This subroutine was used from only 1 place, so it's an easier to move.
Test plan:
Use the membership_expiry.pl cronjob script using the different
options.
The behavior should be the same as prior to this patch.
prove t/db_dependent/Koha/Patrons.t
should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch create a Koha::Acquisition::Booksellers module and
Koha::Acquisition::Bookseller::Contract[s] modules.
All code in the acquisition module is adapted to use the CRUD methods of
Koha::Object[s].
The former C4 routines are removed.
Test plan:
Since a lot of files are impacted by this patch, try a complete
acquisition workflow and try to catch errors.
Be focused on bookseller and bookseller' contacts data.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch makes Koha::Patron::Modification aware of the new extended_attributes
column, which is expected to contain valid JSON data.
The ->store method is modified so it validates the field value (i.e. the content
is decoded using the JSON library) and raises a convenient exception in case of
failure.
This behaviour change is covered by the provided unit tests.
To test:
- Apply the patchset
- Run:
$ prove t/db_dependent/Koha/Patron/Modifications.t
=> SUCCESS: Tests make sense, and they pass
- Sign off :-D
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch can be skipped by the RM, it just updates the DBIC schema files.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Test plan:
prove t/db_dependent/Koha/Patrons.t
should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
If no dateofbirthrequired or upperagelimit is set on the interface, the
->store method will receive an empty string defined for these values.
For INT field, we must explicitely set these empty value to undef
instead to avoid MySQL 5.7 (and strict mode) to raise:
DBD::mysql::st execute failed: Incorrect integer value: ' for column
'dateofbirthrequired''
Test plan:
Using MySQL 5.7 (and/or sql_mode=STRICT_TRANS_TABLES)
Create a patron category without specifying upperagelimit or
dateofbirthrequired
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
When calling the proposed version of get_effective_issuing_rule with undefined
parameter values, a following crash occurs:
SQL::Abstract::puke(): [SQL::Abstract::__ANON__] Fatal: SQL::Abstract before v1.75
used to generate incorrect SQL when the -IN operator was given an undef-containing
list: !!!AUDIT YOUR CODE AND DATA!!! (the upcoming Data::Query-based version of
SQL::Abstract will emit the logically correct SQL instead of raising this
exception) at /home/ubuntu/kohaclone/Koha/Objects.pm line 182
This patch adds a test to cover this problem and fixes the issue.
To test:
1. Run t/db_dependent/Koha/IsssuingRules.t
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch modifies method get_effective_issuing_rule in Koha::IssuingRules
aiming to optimize the search for matching issuing rule.
Before this patch, in worst case scenario, we have had to make a SELECT query
eight times. This will have a negative impact on performance where-ever we need
to find matching issuing rule multiple times, if the search is not directly
matching an issuing rule on the first query.
This patch makes get_effective_issuing_rule have a stable performance on both
best and worst case, whereas the old method was really fast on the best case
and really slow on the worst case.
However, this patch slightly lowers the performance for best case, where matching
issuing rule is found instantly before (branchcode, categorycode and itemtype all
are specifically defined in issuing rules). For all other cases this patch offers
a performance improvement.
To test:
1. Run t/db_dependent/Koha/IssuingRules.t and compare the results with previous
tests.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Returns one and only one object that is part of this set.
Returns undef if there are no objects found.
->single is faster than ->search->next
This is optimal as it will grab the first returned result without instantiating
a cursor.
It is useful for this Bug as we only want to select the top row of found issuing
rules.
To test:
1. Run t/db_dependent/Koha/Objects.t
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch adds a new method Koha::Checkout->is_overdue and provide tests
to cover it.
The goal is to behave like GetItemIssues set the 'overdue' flag to
issues.
I don't understand why the existing GetItemIssues truncate dates to
minutes, so I did not recreate this behavior.
Test plan:
prove t/db_dependent/Koha/Checkouts.t
should return green
Signed-off-by: Mika Smith <mikasmith@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Koha::Issues and Koha::Checkouts have been added to the codebase to
represent the same thing.
In ODLIS the word Issue is never used in the sense we use it. Another
problem with Issue is it has so many meaning in English (such as
problem/bug)
The word Checkout *is* in ODLIS, closer to what we use:
http://www.abc-clio.com/ODLIS/odlis_c.aspx#checkoutslip
Test plan:
git grep Koha::Issue
should not return any occurrences and the tests must still pass
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This method will be useful to get the current holds placed on a given
bibliographic record.
Test plan:
prove t/db_dependent/Koha/Biblios.t
should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Test plan:
prove t/db_dependent/Accounts.t
should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Test plan:
prove t/db_dependent/Koha/Patrons.t
should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This method will be convenient when moving code to Koha::Patrons
Test plan:
prove t/db_dependent/Koha/Patrons.t
should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
As said in the previous commit, I considered SetAge as unnecessary and
removed it.
Test plan:
1/ Edit a patron using the different 'Edit' links
2/ Play with the patron category limited to age ranges, and date of
birth
3/ You should get the expected warning if the date of birth is inside
the patron category date range.
To finish:
prove t/Circulation/AgeRestrictionMarkers.t t/db_dependent/Reserves.t \
t/db_dependent/Koha/Patrons.t t/db_dependent/Members.t
should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Yes, we are fixing these four typos here.
Test plan:
[1] Read the changes.
[2] Run t/Auth_with_shibboleth.t
[3] Run git grep recieved| grep -v -e 'recievedlist' | grep -v -e 'serials-recieve.tt'
Note: serials-recieve.tt is just history. Bonus points for the one who makes
us get rid of that column recievedlist.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
EDIT:
Rebased. Change in Accounts has been corrected already.
Removed the po file.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This is the second patch in a series to unify all payment functions into
a single mathod
Test Plan:
1) Apply this patch
2) prove t/db_dependent/Accounts.t
3) Test fine payment via the "Pay" button
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Followed test plan, works as expected.
Signed-off-by: Marc Veron <veron@veron.ch>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Using a DateTime object with a timezone of America/Santiago
was causing fatal errors for Koha::Calendar->is_holiday
and Koha::Calendar->exception_holidays, when the objects
were truncated to an invalid local time.
Using a floating zone allows us to use the same day, month, year
for comparison purposes without running into the possibility of
creating an invalid local time and thus a fatal software error.
Edit:
While the changes to is_holiday and single_holiday make sense (Jonathan agrees too)
I didn't manage to have them fail, because truncate is not failing in my trials, but
days_between. So to me, it narrows down to have exception_holiday return floating tz
datetime objects so it doesn't break days_between.
Anyway, it is ok to push this patch, and the regression test I provide covers this scenario
I'm describing.
To test:
- Apply the regression tests patch
- Run:
$ prove t/db_dependent/Holidays.t
=> FAIL: Unexpected error due to bad timezone/date combination
- Apply this patch
- Run:
$ prove t/db_dependent/Holidays.t
=> SUCCESS: Tests pass
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This method will be used by several patches later.
Test plan:
prove t/db_dependent/Koha/Patrons.t
should return green
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Number of tests in Patrons.t corrected.
Method is_going_to_expired (no english!) renamed to is_going_to_expire.
Adding a negative duration replaced by a subtract. Reads easier.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
In order to be consistent, we need to create this method as well.
Test plan:
Make sure the pref NotifyBorrowerDeparture works as expected
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Article requsts from OPAC and from intranet are showing an 'internal
server error'. I am testing in 16.11 with Plack.
Test Plan:
1) Set circ rule to record only article requests
2) Attempt to place a request on a record with no items
3) Note the error
4) Apply this patch
5) You should now be able to place the request!
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Elastic appears to be indexing onloan as a string, but our code assumes
it is a boolean.
Test Plan:
1) Ensure you are set up using Elastic as your search engine
2) Search only for available items from the advanced search
3) Note you get no results
4) Apply this patch
5) Re-run the search
6) You should now get results!
Signed-off-by: Jennifer Schmidt <jschmidt@switchinc.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Can't call method "lib" on an undefined value at
/home/vagrant/kohaclone/Koha/AuthorisedValues.pm line 137.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Ok I am silly, we needed to replace to use the cache mechanism for
search_by_koha_field, not find_by_koha_field...
Let's create another subroutine
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Most of the time we just need the descriptions (lib or
opac_description), so let's add a new method for that and cache the
descriptions in L1.
Ideally we should cache it in L2 as well, but the AV code is not robust
enough to allow that
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Marcel de Rooy
Jonathan Druart
Kyle M Hall
Magnus Enger
Tomas Cohen Arazi
Lari Taskula
Julian Maurice
David Cook