Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This commit is generated using:
% perl misc/devel/tidy.pl
*within* ktd, to get the same version of perltidy than what will be used
by our CI (currently v20230309).
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch addresse the fact Koha needs to strip out stray item field in
MARC records coming from the DB. While this is not ideal, it has worked
so far, limiting the negative effects of bad data to Koha users.
I put a FIXME because I think it deserves to be revisited colectively at
some point. I filed bug 38406 to track the discussion around this.
The `marcmarcrecord2csv.t` tests cover this behavior and I thought
removing it was not in the scope of this bug.
I decided to move the removal one step up in the call chain so it
applies to all calls to `$metadata->record`, not only those that ask to
include items. This is the right thing to do while we keep this
behavior.
To test:
1. Run:
$ ktd --shell
k$ prove t/db_dependent/Record/marcrecord2csv.t
=> FAIL: Item information gets extracted from more items than expected
(i.e. the item in the MARC record is not stripped out so 'Withdrawn'
shows 3 times instead of 2).
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests pass!
4. Run:
k$ prove t/db_dependent/Koha/Biblio/Metadata.t
=> SUCCESS: Tests pass! The behavior change in $metadata->record works
as intended!
5. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a method for retrieving the related *Koha::RecordSource* from a *Koha::Biblio::Metadata* object.
The method is covered by tests:
1. Apply this patch
2. Run:
$ ktd --shell
k$ prove t/db_dependent/Koha/Biblio/Metadata.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Sponsored-by: ByWater Solutions
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a convenient method that tells if the record is
editable, according to it's record source.
To test:
1. Apply this patch
2. Run:
$ ktd
k$ prove t/db_dependent/Koha/Biblio/Metadata.t
=> SUCCESS: Tests pass! Use cases covered!
3. Sign off :-D
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Bug 31791: Rename can_be_edited -> source_allows_editing
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Might just be a theoretical thing now, but safer to clone.
Test plan:
Run t/db_dependent/Koha/Biblio/Metadata.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Resolve:
WARN tidiness
The file is less tidy than before (bad/messy lines before: 21, now: 24)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This adds a routine that can strip non xml characters form a record.
It is intended for cases where we do not wish to throw an exception,
but rather need to process a record to allow other work to continue
To test:
prove -v t/db_dependent/Koha/Biblio/Metadata.t
Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
MARC::Record::insert_fields_ordered reverses the order of fields
with the same tag. So we need to reverse the input order here.
This made Exporter/Record.t fail.
Test plan:
Run t/db_dependent/Koha/Biblio/Metadata.t
Run t/db_dependent/Exporter/Record.t (should pass now)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Test plan:
Run t/db_dependent/Koha/Biblio/Metadata.t again. Should pass.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Not sure about the warn, we shouldn't need it as we are raising an
exception. But better (for now) than introducing regressions.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Includes:
Bug 29697: (follow-up) Use flag embed_items
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Bug 23846 added a check for invalid MARCXML in search_for_data_inconsistencies.pl.
But the error message does not show the biblionumber of invalid biblio record.
Test plan:
1) Import a bibliographic record with invalid XML, you can add non printable characters, like 0x1F (CTRL-V 1F with vim)
2) Run misc/maintenance/search_for_data_inconsistencies.pl
3) Check you see correct biblionumber in error message :
Invalid data, cannot decode object (id=xxx, biblionumber=yyy, format=marcxml, schema=zzz, decoding_error=...
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch standardises the encoding name used in direct calls
to new_from_xml() to 'UTF-8' instead of 'utf8' or 'utf-8'.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 9978 should have fixed them all, but some were missing.
We want all the license statements part of Koha to be identical, and
using the GPLv3 statement.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch refactors the original implementation so it is ready for
supporting other formats. It also adds some error handling.
To test:
- Apply this patch
- Run:
$ kshell
k$ prove t/db_dependent/Koha/Biblio/Metadata.t
=> SUCCESS: Tests pass!
- Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
It would be very convenient to be able to retrieve a bib's attached metadata objects directly from the biblio object. This would be very useful for Template Toolkit syntax in slips and notices where a library would like to use arbitrary metadata fields that aren't stored in the database.
Test Plan:
1) Apply this patch
2) prove t/db_dependent/Koha/Biblio.t
Signed-off-by: Te Rahui Tunua <terahuitunua@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Two discussions on koha-devel lead to the same conclusion:
biblioitems.marcxml should be moved out this table
- biblio and biblioitems
http://lists.koha-community.org/pipermail/koha-devel/2013-April/039239.html
- biblioitems.marcxml & biblioitems.marc / HUGE performance issue !
http://lists.koha-community.org/pipermail/koha-devel/2016-July/042821.html
There are several goals to do it:
- Performance
As Paul Poulain wrote, a simple query like
SELECT publicationyear, count(publicationyear) FROM biblioitems GROUP BY publicationyear;
takes more than 10min on a DB with more than 1M bibliographic records
but only 3sec (!) on the same DB without the biblioitems.marcxml field
Note that priori to this patch set, the biblioitems.marcxml was not
retrieved systematically, but was, at least, in
C4::Acquisition::GetOrdersByBiblionumber and C4::Acquisition::GetOrders
- Flexibility
Storing the marcxml in a specific table would allow use to store several
kind of metadata (USMARC, MARCXML, MIJ, etc.) and different formats (marcflavour)
- Clean code
It would be a first step toward Koha::MetadataRecord for bibliographic
records (not done in this patch set).
Test plan:
- Update the DBIC Schema
- Add / Edit / Delete / Import / Export bibliographic records
- Add items
- Reindex records using ES
- Confirm that the following scripts still work:
* misc/cronjobs/delete_records_via_leader.pl
* misc/migration_tools/build_oai_sets.pl
- Look at the reading history at the OPAC (opac-readingrecord.pl)
- At the OPAC, click on a tag, you must see the result
Note: Changes in Koha/OAI/Server/ListRecords.pm is planned on bug 15108.
Signed-off-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Zeno Tajoli <z.tajoli@cineca.it>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Tomas Cohen Arazi
Koha Development Team
Marcel de Rooy
Nick Clemens
Jonathan Druart
Fridolin Somers
Andreas Roussos
Kyle M Hall