This patch adds a new option to allow defining overlay rules for 'Add to basket' - such that they
will be used when adding orders in acquisition. Existing 'Staged MARC Import' rules will be copied to
the new fitler in order to preserve behavior
To test:
1 - Paste the following into MarcFieldsToOrder and save
price: 975$p
quantity: 975$q
budget_code: 975$h
2 - Paste the following into MarcItemFieldsToOrder and save
homebranch: 949$a
holdingbranch: 949$b
itype: 949$y
nonpublic_note: 949$x
public_note: 949$z
loc: 949$c
ccode: 949$8
notforloan: 949$7
uri: 949$u
copyno: 949$t
price: 949$g
replacementprice: 949$v
itemcallnumber: 949$o
quantity: 949$k
budget_code: 949$l
3 - Go to Admin -> Record overlay rules
4 - Define 2 rules:
Source - Staged Marc Import - 949 - Add and append (the rest of the options are set when add and append is selected
Source - Staged Marc Import - * - Protect (the rest of the options are set when add and append is selected
5 - Grab test file from bug 39518
6 - Import the records, ignoring items
7 - Edit the test file and change some values for the 949 rows
8 - Go to Acquisitions, find a basket for a vendor, add to basket from a new file
9 - Stage the file to find matches (already imported above) and to replace
10 - Complete the orders
11 - View the record and confirm the new 949 are added
12 - Edit the 949 in the file again
13 - Apply patch, update database, restart all
14 - Edit overlay rules and remove the 'Staged Marc Import' rules, leaving only the 'Add to basket rules'
15 - Add to basket again using the updated file and confirm new 949 are added
16 - Success!
Signed-off-by: Benjamin Daeuber <BDaeuber@fargolibrary.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This commit is generated using:
% perl misc/devel/tidy.pl
*within* ktd, to get the same version of perltidy than what will be used
by our CI (currently v20230309).
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
1. Go to Cataloging > "Stage records for import".
2. Upload the file "ExportMemento2024061010532869Marc8.mrc" or a MARC8 encoded file.
3. In the form, select the options :
- Record type: Bibliographic
- Character encoding: MARC8
- Format: MARC
4. Click "Stage for import".
5. Find or create an open basket in the Acquisitions module.
6. Click "add to basket".
7. Select "From a staged file" and select the previous staged file.
9. Click "Select all" to check all records.
10. In the item information tab, choose the Document type.
11. Verify the titles of the records in the basket:
--> Koha replaces accents with symbols �
12. Apply the patch.
13. Do the same from step 1 and notice there is no more encoding issues.
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Add unit tests for authorities and deletion
Fix dbrev and kohastructure discrepancies
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Rename Oaipmh to OAI
Rename oaiservers to oai_servers
Rename OaiServer to OAIServer
Rename oai_servers.id to oai_servers.oai_server_id
Rename ServerSearch to server_search
Use a callback function for logs
Use context for string "Set"
Split translation string
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test plan:
- Run updatedatabase.pl
- Apply the DBIC update patch
- Setup an OAI repository in Administration -> OAI repositories (admin/oaiservers.pl)
If the repository dataformat is in oai_dc, you can use the sample XSLT files:
For Unimarc:
OAIDCtoUNIMARCXML.xsl for bibliographic records
AuthOAIDCtoUNIMARCXML.xsl for authorities
For Marc21:
OAIDCtoMARC21XML.xsl for bibliographic records
AuthOAIDCtoMARC21XML.xsl for authorities
If the repository dataformat is in marc-xml, you can use
OAIMarcxml2KohaMarcxml.xsl for both Unimarc and Marc21.
- Some repositories to test with:
Bibliographic records, oai_dc:
Endpoint: http://staroai.theses.fr/OAIHandler
Set: ddc:260
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/OAIDCtoUNIMARCXML.xsl
or
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/OAIDCtoMARC21XML.xsl
Authorities, oai_dc:
Endpoint: https://www.idref.fr/OAI/oai.jsp
Set: a
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/AuthOAIDCtoUNIMARCXML.xsl
or
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/AuthOAIDCtoMARC21XML.xsl
Authorities, marc-xml, Unimarc:
Endpoint: https://www.idref.fr/OAI/oai.jsp
Set: a
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/OAIMarcxml2KohaMarcxml.xsl
- Note: marc-xml harvesting will not work with HTTP::OAI::Harvester version 3.xx
Version 4.xx is needed.
See Bug 17704 for updating Koha default version to 4.xx
- If you want to receive a report email, fill the OAI-PMH:HarvestEmailReport
syspref. The model used (OAI_HARVEST_REPORT) can be edited in tools -> Notices and slips.
- start a harvest with the following command:
misc/cronjobs/harvest_oai.pl
This script starts an OAI Harvest
This script has the following parameters :
-h --help: this message
-v --verbose (enable output to stdout)
-r --repository: id of the OAI repository
-d --days: number of days to harvest from (optional)
-l --list: list the OAI repositories
-f --force: force harvesting (ignore records datestamps) (optional)
Example: misc/cronjobs/harvest_oai.pl -r 1 -d 10 -v -f
- Check the logs in the Log Viewer (tools/viewlog.pl)
(CronjobLog syspref must be activated)
Sponsored-by: KohaLa
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Making it more visible that this also applies to authority
records.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds a replace routine to Koha::Import::Record - largely
copying and updating code from C4::ImportBatch
To test:
prove t/db_dependent/Koha/Import/Records.t
Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch:
1 - Adds an atomic update to add a primary key to import_auths table
2 - Adds objects for Koha::Import::Records::Auths
3 - Adds tests for import auth and biblio objects
Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Thispatch adds the new method and alters addorderiso2907.pl to use this
rather than GetRecordImportMatches
To test:
1 - Import the attached record several times
2 - Set up a matching rule:
TitleAuthor threshold: 100
Matchpoint:
search index: title, score: 100, tag: 245$a
search index: author, score: 100, tag:100$a
3 - Edit one of the imported records to have a different author
4 - Stage the file again. and match using the matchpoint above
5 - Note that matches are found and listed on batch management, with the lowest scored match last
6 - Choose that match
7 - In acquisitions, add to a basket from the staged file
8 - Check the box for the record
9 - Note the match lists the biblionumber for the highest scoring match, not the chosen one
10 - Add an order and note it is for the wrong biblio
11 - Appy patch
12 - Restart_all
13 - Stage the file again and choose a lower scoring match
14 - Confirm when adding to basket this match is preserved
15 - Complete order and verify correct biblio ordered
16 - Stage again, select no match
17 - Confirm no match listed when adding to basket, and choose 'Do not look for matching records' while adding
18 - Confirm order is created on a new biblio
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch removes the now unused: _update_import_record_marc
Additionally, as items are already present in import biblio we no longer need to embed
them, so that parameter is removed and the option removed from the sub and pod and everywhere
it was used
In all cases, we were embedding, so we don't need a way to get without
Tests updated
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
MARC::Record and MARC::File::* modules sometimes use the position 09 of
the leader to detect encoding. A blank character means 'MARC-8' while an
'a' means 'UTF-8'.
In a UNIMARC authority this position is used to store the authority type
(see https://www.transition-bibliographique.fr/wp-content/uploads/2021/02/AIntroLabel-2004.pdf [FR]).
In this case, 'a' means 'Personal Name'.
The result is that the import will succeed for a Personal Name
authority, but it will fail for all other authority types.
Steps to reproduce:
0. Be sure to have a Koha UNIMARC instance.
1. Download the MARCXML for "Honoré de Balzac"
curl -o balzac.marcxml https://www.idref.fr/02670305X.xml
2. Verify that it's encoded in UTF-8
file balzac.marcxml
(should output "balzac.marcxml: XML 1.0 document, UTF-8 Unicode
text")
3. Go to Tools » Stage MARC for import and import balzac.marcxml with
the following settings:
Record type: Authority
Character encoding: UTF-8
Format: MARCXML
Do not touch the other settings
4. Once imported, go to the staged MARC management tool and find your
batch. Click on the authority title "Balzac Honoré de 1799-1850" to
show the MARC inside a modal window. There should be no encoding
issue.
5. Write down the imported record id (the number in column '#') and go
to the MARC authority editor. Replace all URL parameters by
'breedingid=THE_ID_YOU_WROTE_DOWN'
The URL should look like this:
/cgi-bin/koha/authorities/authorities.pl?breedingid=198
You should see no encoding issues. Do not save the record.
6. Import the batch into the catalog. Verify that the authority record
has no encoding issue.
7. Now download the MARCXML for "Athènes (Grèce)"
curl -o athènes.marcxml https://www.idref.fr/027290530.xml
8. Repeat steps 2 to 6 using athènes.marcxml file. At steps 4 and 5 you
should see encoding issues and that the position 9 of the leader was
rewritten from 'c' to 'a'. Strangely, importing this batch fix the
encoding issue, but we still lose the information in position 09 of
the leader
This patch makes use of the MARCXML representation of the record instead
of the ISO2709 representation, because, unlike
MARC::Record::new_from_usmarc, MARC::Record::new_from_xml allows us to
pass directly the encoding and the format, which prevents data to be
double encoded when position 09 of the leader is different that 'a'
Test plan:
- Follow the "steps to reproduce" above and verify that you have no
encoding issues.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
To test:
prove -v t/db_dependent/Koha/Import/Record/Items.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Before dropping a primary key we need to see if one exists - this updates
the installer method to allow searching for existence of key
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Bug 26326: (follow-up) Address qa script failures
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Nick Clemens
Tomás Cohen Arazi
Koha Development Team
Hammat Wele
Matthias Meusburger
Marcel de Rooy
Julian Maurice