Commit Graph

130 Commits

Author SHA1 Message Date
Jonathan Druart 01bba47068
Bug 29486: Add check in search_for_data_inconsistencies.pl
If the MARC record does not contain the correct biblionumber of
biblioitemnumber, the script will display the following warning:

== Bibliographic records have MARCXML without biblionumber or biblioitemnumber ==
        * Biblionumber 4242 has '1' in 999$c
        * Biblionumber 4242 has biblioitemnumber '4242' but should be '1' in 999$d
=> The bibliographic records must have the biblionumber and biblioitemnumber in MARCXML

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-06-12 12:36:00 -03:00
Fridolin Somers 194414625d
Bug 33603: Fix misc/maintenance/search_for_data_inconsistencies.pl when biblio.biblionumber on control field
Since bug 29486, misc/maintenance/search_for_data_inconsistencies.pl search for biblio.biblionumber in MARC record with $record->subfield().
This fails when field is a control field (< 10).
Idem for biblioitems.biblioitemnumber

Test plan :
1.0) On a UNIMARC database (biblio.biblionumber is on 001)
1.1) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Without patch you get error : Control fields (generally, just tags below 010) do not have subfields, use data()
=> With patch no error
2.0) On a MARC21 database (biblio.biblionumber is on 999c)
2.1) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Check you see no error

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit cc1695ba96)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
(cherry picked from commit 9971011fb9)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 6254b429b6)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>
2023-05-23 20:43:31 +02:00
Tomas Cohen Arazi 92ad549936 Bug 33341: Address some perlcritic errors in 5.36
Some old-style code is making our tests fail when run in Debian Testing.

This patch addresses this.

To test:
1. Launch bookworm KTD:
   $ KOHA_IMAGE=master-bookworm ktd up -d
2. Run:
   $ ktd --shell
  k$ prove t/00-testcritic.t
=> FAIL: It fails!
3. Apply the patch
4. Repeat 2
=> SUCCESS: Tests now pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 10d12f999f)
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
(cherry picked from commit c1c5420afa)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 548a3325ba)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>
2023-05-03 21:03:35 +02:00
Fridolin Somers 0f5208ebe1
Bug 32678: Add new line in authorized values tests in search_for_data_inconsistencies.pl
In search_for_data_inconsistencies.pl, the test for authorized values is a list in one line :
	* The Framework *VR* is using the authorised value's category *LOC*, but the following items.location do not have a value defined ({itemnumber => value }):
 {94 => AV} {95 => AV} {96 => AV} {97 => AV} {98 => AV} {99 => AV} {100 => AV} {101 => AV} {102 => AV} {103 => AV}

It would be more clear with new lines, especially for scripts (grep, awk ...) :
	* The Framework *VR* is using the authorised value's category *LOC*, but the following items.location do not have a value defined ({itemnumber => value }):
 {94 => AV}
 {95 => AV}
 {96 => AV}
 {97 => AV}
 {98 => AV}
 {99 => AV}
 {100 => AV}
 {101 => AV}
 {102 => AV}
 {103 => AV}

Test plan :
1) In koha-testing-docker
2) Delete in authorized values LOC the value AV
3) Run misc/maintenance/search_for_data_inconsistencies.pl
=> You see the new line in result

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 45323886ae)
Signed-off-by: Jacob O'Mara <jacob.omara@ptfs-europe.com>
(cherry picked from commit 1453472e93)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 519b4f2b59)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>
2023-03-29 14:53:57 +02:00
Fridolin Somers 5a5595483e Bug 31299: Fix duplicate output in search_for_data_inconsistencies.pl
In search_for_data_inconsistencies.pl when there are several inconsistencies on same framework, the second output of items contains the first one.

Looks like it is since Bug 21466

Test plan using koha-testing-docker :
1) Create 2 inconsistencies on the same item via SQL :
   update items set itemlost = 9 where itemnumber=900;
   update items set notforloan = 8 where itemnumber=900;
2) Without patch
3) Run ./misc/maintenance/search_for_data_inconsistencies.pl
=> You see duplicate output :
== Wrong values linked to authorised values ==
	* The Framework *BKS* is using the authorised value's category *NOT_LOAN*, but the following items.notforloan do not have a value defined ({itemnumber => value }):
 {900 => 8}
	* The Framework *BKS* is using the authorised value's category *LOST*, but the following items.itemlost do not have a value defined ({itemnumber => value }):
 {900 => 8} {900 => 9}

4) Apply patch
5) Run ./misc/maintenance/search_for_data_inconsistencies.pl
=> Fixed :D
== Wrong values linked to authorised values ==
	* The Framework *BKS* is using the authorised value's category *LOST*, but the following items.itemlost do not have a value defined ({itemnumber => value }):
 {900 => 9}
	* The Framework *BKS* is using the authorised value's category *NOT_LOAN*, but the following items.notforloan do not have a value defined ({itemnumber => value }):
 {900 => 8}

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 3d0015fe69)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e731d4cff4)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>
2022-11-14 17:05:26 +01:00
Marcel de Rooy f4e145bf01 Bug 31239: (QA follow-up) Fixing ternary formatting
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 4481817701)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit fd2bc6b897)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>
2022-11-14 15:24:21 +01:00
Jonathan Druart da04cc1fac Bug 31239: Fix incorrect AV linked to a mapped biblio column
If an AV is linked to a MARC field mapped with a biblio column, the
search_for_data_inconsistencies.pl script might explode with

The method Koha::Biblioitem->title is not covered by tests!

Trace begun at /kohadevbox/koha/Koha/Object.pm line 875
Koha::Object::AUTOLOAD('Koha::Biblioitem=HASH(0x556b67fa7168)') called at misc/maintenance/search_for_data_inconsistencies.pl line 246

Test plan:
For a given framework, pick a biblio using it
Link 245$a with an authorised value category
Run the script
=> Notice that with this script applied you will see the warning
=> Without this patch you got the error

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 54711942f1)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 2e363d0d0c)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>
2022-11-14 15:24:21 +01:00
Caroline Cyr La Rose 2cc8e5e353 Bug 31155: Document --since option in help of borrowers-force-messaging-defaults.pl
This patch adds a section about the --since option in the help section
of borrowers-force-messaging-defaults.pl

To test:
0) Apply patch
1) run the script with the --help option, e.g.
./misc/maintenance/borrowers-force-messaging-defaults.pl --help
--> There should be an explanation of the --since option with examples
    for specific and relative dates
2) Optionally, run the script with different options
--> The behaviour shouldn't have changed

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 42b3169120)

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 9db4824675)
Signed-off-by: Arthur Suzuki <arthur.suzuki@biblibre.com>
2022-10-12 15:47:56 +02:00
Nick Clemens c7eb047126 Bug 28352: Only check authorised values mapped to DB fields
The errors reported seem to be caused by authorised values mapped to MARC fields
but not mapped to a koha field.

We should additionally make sure to check the Default framework

Also, adding comment to indicate we only check records with items, because we do

TO test:
1 - In a framework that is not the default map a MARC field to an authorised value, but not a koha field
2 - In SQL, force the kohafield to NULL for the mapping you just make
    UPDATE marc_subfield_structure SET kohafield = NULL WHERE frameworkcode='BKS' and authorised_value='HINGS_AS'
3 - perl misc/maintenance/search_for_data_inconsistencies.pl
4 - get the following errors:
Use of uninitialized value $tmp_kohafield in pattern match (m//) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 151.
Use of uninitialized value $tmp_kohafield in substitution (s///) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 154.
Can't call method "get_column" on an undefined value at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 157.
5 - Apply patch
6 - Repeat
7 - No more errors

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-01 16:28:55 +02:00
Marcel de Rooy a0ebdab046 Bug 29078: Resolve division by zero in touch_all scripts
Illegal division by zero at misc/maintenance/touch_all_biblios.pl line 102.

Trivial fix.
Funny that the percentage is not even shown without verbose flag ;)

Test plan:
Run misc/maintenance/touch_all_biblios.pl -where 'biblionumber<0'
You should now see:
Good: 0, Bad: 0 (of 0) in 0 seconds

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-09-28 15:12:45 +02:00
Jonathan Druart 9d6d641d1f Bug 17600: Standardize our EXPORT_OK
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.

That way we will need to explicitely define the subroutine we want to
use from a module.

This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests

And a lot of other manual changes.

export.pl is a dirty script that can be found on bug 17600.

"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;

The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules

Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).

EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.

@EXPORT and @EXPORT_OK are the two main variables used during export operation.

@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.

@EXPORT_OK does export of symbols on demand basis.
"""

If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
  - use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-16 08:58:47 +02:00
Jonathan Druart 41a8005d10 Bug 28606: Remove $DEBUG and $ENV{DEBUG}
We should remove the debug statements or use Koha::Logger when we want
to keep it.

Test plan:
Confirm that occurrences of remaining occurrences of DEBUG need to be
kept (historical scripts for instance)
Confirm that the occurrences removed by this patch can be removed
Confirm that the occurrences replaced by Koha::Logger are correct

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Looks good to me, noting a few minor points on BZ.

JD amended patch: replace "warn #Finished" with "#warn Finished", and
put the statement on a single line

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-06-24 11:53:44 +02:00
Jonathan Druart 8f071b01f5 Bug 27963: (bug 23463 follow-up) Fix touch_all_items.pl
This script is used to fix inconsistencies in the items table. Prior to
bug 23463, ModItem dealt with that.

We need to make some attributes as dirty to make sure the
Koha::Item->store method will fix them.

Test plan:
Be familiar with how touch_all_items is supposed to work and confirm
that the different inconsistencies are fixed.

Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-04-01 17:13:56 +02:00
Jonathan Druart 46f7239b08 Bug 27673: Replace YAML with YAML::XS
From tht YAML pod:

"""
This module has been released to CPAN as YAML::Old, and soon YAML.pm will be changed to just be a frontend interface module for all the various Perl YAML implementation modules, including YAML::Old.

If you want robust and fast YAML processing using the normal Dump/Load API, please consider switching to YAML::XS. It is by far the best Perl module for YAML at this time. It requires that you have a C compiler, since it is written in C.
"""

See also
https://gitlab.com/koha-community/qa-test-tools/-/merge_requests/35

Test plan:
Try some place where YAML::XS is not used and confirm that it works
correctly

QA note: This patch removes some uses of YAML that were not useful

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-02-16 14:54:50 +01:00
Didier Gautheron 10c08b6552 Bug 25381: XSLTs should not define entities
Bug 23290 explicitly disabled entity expansion during XML parsing for security reasons.

However, many XSLTs define the following entity:
<!ENTITY nbsp "&#160;" >

They don't use the entity &nbsp, but its presence could lead to confusion.

Signed-off-by: Eden Bacani <eden.bacani@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-22 13:46:40 +01:00
Jonathan Druart 67b1b20e33 Bug 24272: Correctly encode output to prevent 'Wide character in say'
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-20 13:42:08 +01:00
Jonathan Druart b191529344 Bug 24272: Use Pod2Usage and surround values with ''
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-20 13:42:08 +01:00
Nick Clemens a9ea5d1ba4 Bug 24272: (follow-up) Test defined not true
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-20 10:16:48 +01:00
Nick de5527f37d Bug 24272: add check_sysprefs_cache.pl
This script ensure that no sysprefs have been changed directly in the database and/or
that the cache has not become corrupted. We have occasionally seen this happen on production sites

To test:
1 - In the staff interface go to Administration
2 - Search for system preference 'IntranetUserJS'
3 - Add content to the syspref:
    console.log('Hi!');
4 - On the command line launch mysql
    sudo koha-mysql kohadev
5 - Alter the syspref directly
    UPDATE systempreferences SET value = "console.log('Bye!');" WHERE variable = 'IntranetUserJS';
6 - run the script
    perl misc/maintenance/check_syspref_cache.pl
7 - You are warned about the altered system preference

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-20 10:16:48 +01:00
Kyle M Hall 4f112b782b Bug 27276: Fix syntax error
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-04 16:33:15 +01:00
Jonathan Druart f228c9ee9f Bug 27276: Prevent borrowers-force-messaging-defaults.pl to crash
MySQL 5.7 does not allow queries using 0000-00-00

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-04 16:33:15 +01:00
Julian Maurice 96cc447045 Bug 25898: Prohibit indirect object notation
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-15 12:56:30 +02:00
Fridolin Somers 15174e7bd3 Bug 26407: fix query in 'title exists' in `search_for_data_inconsistencies.pl`
Bug 23871 added check for 'title exists' to `search_for_data_inconsistencies.pl`.
There is a typo in DBIx query to create or : hashref instead of arrayref.

Test plan :
1) Change title of biblio record XXX in SQL :
   UPDATE biblio SET title = '' WHERE biblionumber=XXX
2) Change title of biblio record YYY in SQL :
   UPDATE biblio SET title = NULL WHERE biblionumber=YYY
3) Run misc/maintenance/search_for_data_inconsistencies.pl
4) Check you see both records

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-18 10:37:56 +02:00
Jonathan Druart f1f9c6dc74 Bug 26384: Fix executable flags
.pm must not have -x
.t must have -x
.pl must have -x

Test plan:
Apply only the first patch, run the tests and confirm that the failures
make sense
Apply this patch and confirm that the test now returns green

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-11 09:56:56 +02:00
Nick Clemens 005219ea6f Bug 25955: Fix getting index_name
To test:
1 - Have a site running ES
2 - perl misc/maintenance/compare_es_to_db.pl
3 - Lots of output, all bad
4 - Apply patch
5 - Repeate
6 - Whew, looks correct

Signed-off-by: Didier Gautheron <didier.gautheron@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-22 10:17:53 +02:00
Julian Maurice b168f4a2e9 Bug 21395: Make perlcritic happy
This patch adds a .perlcriticrc (copied from qa-test-tools) and fixes
almost all perlcrictic violations according to this .perlcriticrc
The remaining violations are silenced out by appending a '## no critic'
to the offending lines. They can still be seen by using the --force
option of perlcritic
This patch also modify t/00-testcritic.t to check all Perl files using
the new .perlcriticrc.
I'm not sure if this test script is still useful as it is now equivalent
to `perlcritic --quiet .` and it looks like it is much slower
(approximatively 5 times slower on my machine)

Test plan:
1. Run `perlcritic --quiet .` from the root directory. It should output
   nothing
2. Run `perlcritic --quiet --force .`. It should output 7 errors (6
   StringyEval, 1 BarewordFileHandles)
3. Run `TEST_QA=1 prove t/00-testcritic.t`
4. Read the patch. Check that all changes make sense and do not
   introduce undesired behaviour

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-29 12:37:02 +02:00
Nick Clemens 6beda5cf1d Bug 21591: Check for record level item type issues too
To test:
1 - sudo koha-mysql kohadev
2 - UPDATE biblioitems SET itemtype = NULL where biblionumber = 1
3 - UPDATE items SET itype = NULL where biblionumber = 1
4 - perl misc/maintenance/search_for_data_inconsistencies.pl
5 - Notice warnings
6 - Apply patch
7 - Undefined itemtype on bibliolevel is now warned
7 - Test also with itype=""

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-15 11:53:36 +02:00
Katrin Fischer 8cfe35b044
Bug 25082: Make 041 descriptions display, if authorised value list is used
When 041$a or another subfield is linked to an authorised value list,
Koha will replace the codes with the descriptions from the authorised
value in the MARCXML before passing it to the XSLT.

The XSLT was made to translate codes into descriptions - as there is
no code now, this fails and 'unknown language' is shown.

The patch changes the handling so that when the XSLT is handled something
that doesn't match a known code, it displays the subfield without
processing.

To test:
- Create some records with 041.
  Examples: http://www.loc.gov/marc/bibliographic/bd041.html
- Verify this displays nicely in staff and OPAC
- Apply patch
- Verify display is unchanged
- Remove the patch
- Change one of your MARC frameworks (use another framework than
  for your first test records)
  and link the 041 subfields to the LANG authorised value list
- Go to administration and create some entries for LANG with language
  codes and descriptions
- Catalog a 041 using the normal editor (not advanced)
- You can now use a pull down to select the values you want
- Verify the display is now 'Unknown language' for this record
- Apply patch
- Verify all records display the language descriptions now, using
  an authorised value or not
- Run misc/maintenance/generate_MARC21Languages.pl

Verify the output at the end reads like:
<xsl:otherwise>
    <!-- when 041\$a is mapped to an authorised value list, this will show the description -->
    <xsl:value-of select="\$code" />
</xsl:otherwise>

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-05-11 09:46:02 +01:00
Fridolin Somers ec6c143f1b
Bug 23871: data inconsistencies - check title exists
In biblio records, a field must be linked to biblio.title.
200$a in UNIMARC, 100$a MARC21 (and other).

If this field is undefined, some pages like checkouts table can fail.

Test plan :
1) Remove biblio title in SQL :
   UPDATE biblio SET title='' WHERE biblionumber=XXX
2) Run misc/maintenance/search_for_data_inconsistencies.pl
3) You see the record as an inconsistency
4) Reset a title
   UPDATE biblio SET title='50 shades of Grey' WHERE biblionumber=XXX
5) Run misc/maintenance/search_for_data_inconsistencies.pl
6) Record is no longer an inconsistency

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-14 16:55:13 +01:00
Jonathan Druart bbb504c86f
Bug 23463: Replace ModItem with Koha::Item->store
Starting to replace the ModItem calls with Koha::Item->store

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-23 09:26:30 +00:00
Jonathan Druart 7d8b96803f
Bug 24545: Fix license statements
Bug 9978 should have fixed them all, but some were missing.
We want all the license statements part of Koha to be identical, and
using the GPLv3 statement.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-24 13:31:26 +00:00
Jonathan Druart 14d1804652
Bug 21466: Keep all AVs for a given framework
There was a major (and silly) issue in the previous version, only one AV
was kept as we erased the hash value at the end of each iteration:
$invalid_locations_per_framework->{$framework->frameworkcode } =
  { items => $items, av_category => $mss->authorised_value, kohafield => $kohafield };

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-02-04 13:30:33 +00:00
Martin Renvoize 56fcf5a34d
Bug 22831: (RM follow-up) Code golf
Rather than performing a symmetric diff and then splitting the results
in a further loop this patch changes to logic to use two asymetric diffs
to get the results more directly.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 10:22:13 +00:00
Nick Clemens e9f7a0dd1c
Bug 22831: (follow-up) Sort results, group by db
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 10:11:28 +00:00
Nick Clemens d5c79c201d
Bug 22831: (follow-up) Determine where the record is missing and provide link, show counter
Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 10:11:22 +00:00
Nick Clemens 2b32aee825
Bug 22831: Elasticsearch - add a maintenance script for checking DB vs index counts
This script uses Array::Utils and adds a dependency

To test:
 1 - Have Koha with ES running
 2 - Delete some records from ES
     curl -X DELETE "es:9200/koha_kohadev_biblios/data/5"
     curl -X DELETE "es:9200/koha_kohadev_authorities/data/5"
 3 - perl misc/maintenance/compare_es_to_db.pl
 4 - Note you are notified of problems in both indexes
 5 - perl misc/search_tools/rebuild_elastic_search.pl  -a
 6 - perl misc/maintenance/compare_es_to_db.pl
 7 - Note you are only notified about problems in biblios (assuming you don't have other issues)
 8 - perl misc/search_tools/rebuild_elasticsearch.pl  -b
 9 - perl misc/maintenance/compare_es_to_db.pl
10 - Both counts match, no problems

Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-23 10:11:09 +00:00
Jonathan Druart 4e8ecf0d3d
Bug 21466: Catch any values that should be defined in the AV but are not
Let's expand the idea of the first patch to catch the wrong values for
any others AV.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-13 13:57:52 +00:00
Jonathan Druart e48d396002
Bug 21466: Search for items.location inconsistencies
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-13 13:57:42 +00:00
Jonathan Druart cce358f9af
Bug 23846: Handle exception gracefully at the OPAC
I do not think we should have the same trick as the intranet, and
display a message. This should be enough.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-13 08:04:23 +00:00
Jonathan Druart 1e1c5d17a2
Bug 23846: Add a check to the data inconsistencies script
This may be quite long for big catalogue, but I think it is a good one
to have.

Test plan:
Same as first patch, then execute search_for_data_inconsistencies.pl
Notice the error.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-11-13 08:04:20 +00:00
Pasi Kallinen fc53c88ad4
Bug 20364: Show languages from MARC21 field 041 in intranet
Show the language names coded in field 041, in intranet search
results and in detail view.

Test plan:
1) Apply patch
2) Do a search, and check the results look correct
3) Check some biblio detail pages that they look correct
4) Add some 041 subfields (a, b, d, h, j) in any combinations
5) Repeat 2 and 3

Signed-off-by: Pasi Kallinen <pasi.kallinen@joensuu.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-07-15 11:28:07 +01:00
Martin Renvoize d2e189ca1c Bug 22600: Set 'commandline' interface appropriately
This patch change Koha::Cron to be a more generic Koha::Script class and
update all commanline driven scripts to use it.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-04-10 19:43:11 +00:00
Martin Renvoize e8eb0cf0f2 Bug 22518: Remove references to 'O' accounttype
I went back over 12 years to and still only found "FIXME: What are these
accounttypes" concerning the 'O' type and I couldn't find anywhere where
it was being set.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-03-29 11:38:12 +00:00
Josef Moravec bb231e6208 Bug 21683: (follow-up) Polish the change - based on feedback
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-03-21 18:19:22 +00:00
Josef Moravec 6e3da5397e Bug 21683: Remove accountlines.accountno
Test plan:
1) Play with fines, should work OK
2) Try to print receipts on fines - prinfeercpt.pl, printinvoice.pl
3) git grep getnextacctno -> no occurences
4) git grep accountno should return only:
  installer/data/mysql/atomicupdate/bug_21683_remove_column_accountno.perl
  installer/data/mysql/update22to30.pl
  misc/release_notes/release_notes_3_10_0.txt
  misc/release_notes/release_notes_3_22_0.txt
5) prove
  t/db_dependent/Accounts.t
  t/db_dependent/ILSDI_Services.t
  t/db_dependent/Stats.t
  t/db_dependent/Koha/Account.t

Rescued-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-03-21 18:19:22 +00:00
Jonathan Druart d7c0547bd2 Bug 21846: Fix pod2usage return values
According to existing pattern.
However I am not sure it makes sense to return an errno (and print to STDERR)
if --help is passed.

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-03-15 12:27:44 +00:00
Tomas Cohen Arazi 293eee9959 Bug 21846: (follow-up) Add maintenance script
This patch adds a maintenance script that generates the missing
tags_approval entries based on the tags_all table, and then recalculates
the weights for both tags_approval and tags_index tables.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-03-15 12:27:44 +00:00
Tomas Cohen Arazi fd30cac85a Bug 22155: Adapt uses of biblio_metadata.marcflavour to schema
This patch makes the code using Koha::Biblio::Metadata->marcflavour use
->schema instead for all interactions.

To test:
- Update the DB structure:
  $ updatedatabase
- Update the schema files:
  $ dbic
- Notice all the places in which biblio_metadata is used
  $ cd kohaclone
  $ git grep biblio_metadata
=> SUCCESS: They all use `schema` instead of marcflavour
- Notice all the places that use Koha::Biblio::Metadata:
  $ git grep Koha::Biblio::Metadata
=> SUCCESS: They all use the schema attribute when they used to use
marcflavour
- Run all the modified tests and scripts
=> SUCCESS: We are all good
- Sign off :-D

Note: while this seems like a minor change, the places in which plain
SQL is used really require understanding the queries and how they are
used, because some query results might be passed to some other method
that in turn uses the marcflavour attribute. I of course took that into
account but errare humanum est :-D

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-01-30 11:35:34 +00:00
Jonathan Druart 76141a352e Bug 21150: add the biblionumber to the output
Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-08-22 12:48:22 +00:00
Jonathan Druart 6fd1a8c114 Bug 21150: Search for item types inconsistencies
search_for_data_inconsistencies.pl will now display errors if:
1.item-level_itypes is set to "specific item" and items.itype is not set
or not set to an item type defined in the system (itemtypes.itemtype)
2.item-level_itypes is set to "biblio record" and biblioitems.itemtype is not set
or not set to an item type defined in the system (itemtypes.itemtype)

Test plan:
Use the script and the different possible combinations to display the
errors

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-08-22 12:48:22 +00:00