Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
When responding to ListRecords and ListIdentifiers verbs, OAI server doesn't
return proper resumption token. At the end of a result set, OAI server
generates a resumption token even if there isn't anymore records. Consequently,
OAI harverster will send a new request, based on this invalid resumption,
token. OAI Server responds with an empty resultset, which is considered as an
invalid response by most of the harvesters.
TO TEST:
- Find in your DB, a day where a few biblio records have been created. The
number of created biblios must inferior to OAI-PMH:MaxCount.
- Let say this day is 2014-01-09. Send an OAI-PMH request to Koha OAI Server:
/cgi-bin/koha/oai.pl?verb=ListRecords&metadataPrefix=marcxml&from=2014-01-09&until=2014-01-09
- At the end of the result, you will see a resumption token which looks like that:
<resumptionToken cursor="47">marcxml/47/2014-01-09/2014-01-09/</resumptionToken>
This is wrong. No resumptiion token should be sent since there isn't anymore
records to harvest.
- Apply the patch.
- Resend the OAI-PMH request. There is no resumption token at the end of the
result.
- You could test also with ListIdenfiers verb in place of ListRecord.
Signed-off-by: Christophe Brocquet <christophe.brocquet@obspm.fr>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch changes the value separator in OAI-PMH resumption tokens
from colons to slashes, so that the token string isn't split incorrectly
when a time is included.
TEST PLAN:
1) Turn on the OAI-PMH server syspref in Koha
2) Send a ListRecords request using 'from' and 'until' arguments that
include times (Best to use very far apart times so that you retrieve
more than 50 records which will likely be the trigger for a resumptionToken).
Here is an example:
http://KOHAINSTANCE/cgi-bin/koha/oai.pl?verb=ListRecords&
metadataPrefix=oai_dc&from=2012-09-05T13:44:33Z&until=2014-09-05T13:44:33Z
N.B. Replace KOHAINSTANCE with the URL of your Koha instance.
3) Scroll down to the bottom of the page until you find the resumptionToken.
It will look similar to this:
<resumptionToken cursor="50">
oai_dc:50:2012-09-05T13:44:33Z:2014-09-05T13:44:33Z:
</resumptionToken>
4) Copy that resumption token and send a request with it like so:
http://KOHAINSTANCE/cgi-bin/koha/oai.pl?verb=ListRecords&
resumptionToken=oai_dc:50:2012-09-05T13:44:33Z:2014-09-05T13:44:33Z:
5) The page should (incorrectly) show no records.
6) APPLY PATCH
7) Repeat steps 2, 3, and 4
8) Note that the resumptionToken now uses slashes (e.g. /) instead of
colons.
Note also that now the second request will show records!!!
N.B. This will only happen if Koha has enough records to serve to you.
If your Koha has less than 50 records, try lowering the number provided
in the "OAI-PMH:MaxCount" system preference.
Signed-off-by: Petter Goksoyr Asen <boutrosboutrosboutros@gmail.com>
I understand; I can now confirm the behaviour described in the test plan.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes test plan, all tests and QA script.
Resumption Token works correctly after applying the patch.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
New sql tables:
- oai_sets: contains the list of sets, described by a spec and a name
- oai_sets_descriptions: contains a list of descriptions for each set
- oai_sets_mappings: conditions on marc fields to match for biblio to be
in a set
- oai_sets_biblios: list of biblionumbers for each set
New admin page: allow to configure sets:
- Creation, deletion, modification of spec, name and descriptions
- Define mappings which will be used for building oai sets
Implements OAI Sets in opac/oai.pl:
- ListSets, ListIdentifiers, ListRecords, GetRecord
New script misc/migration_tools/build_oai_sets.pl:
- Retrieve marcxml from all biblios and test if they belong to defined
sets. The oai_sets_biblios table is then updated accordingly
New system preference OAI-PMH:AutoUpdateSets. If on, update sets
automatically when a biblio is created or updated.
Use OPACBaseURL in oai_dc xslt
use encoding(UTF-8) rather than utf-8 for stricter
encoding
Marking output as ':utf8' only flags the data as utf8
using :encoding(UTF-8) also checks it as valid utf-8
see binmode in perlfunc for more details
In accordance with the robustness principle input
filehandles have not been changed as code may make
the undocumented assumption that invalid utf-8 is present
in the imput
Fixes errors reported by t/00-testcritic.t
Where feasable some filehandles have been made lexical rather than
reusing global filehandle vars
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Based on patch by Tomás Cohen Arazi <tomascohen@gmail.com>,
revised to work regardless of the installation mode.
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Magnus Enger <magnus@enger.priv.no>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
- Add preference OAI-PMH:ConfFile. I just add it in web-services.pref
and not in DB. It's enough. It's not an end-user preference. Without
this pref, OAI server operates as previously. And preferences editor
allow to add a new value to the DB if necessary.
- Fix response to ListMetadataFormats which was empty in extended mode.
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Currently Koha OAI server returns records in two formats: marcxml and
oai_dc (Dublin Core). This patch adds a new mode of operation where as
many as necessary metadata formats can be added via XSLT.
Documentation: See the end of oai.pl file to have an explanation of
how it works.
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
The optional description element of an Identify response
can't just be a string. Identify.description is a container
for one or more elements; see http://www.openarchives.org/OAI/2.0/guidelines.htm
For now, simply commenting it out.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
When retrieving a record via the OAI-PMH interface, if one of the fields used
to prepare the DC metadata is not defined in the MARC framework (e.g.,
biblioitems.publicationyear in the default MARC21 framework), an OAI GetRecord
can fail with the following error:
> Can't call method "as_string" on an undefined value at
> /usr/share/koha/opac/cgi-bin/opac/oai.pl line 59.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Jonathan Druart
Frédéric Demians
David Cook
Chris Cormack
Julian Maurice
Colin Campbell
Galen Charlton
Nahuel ANGELINETTI
Galen Charlton
Philippe Jaillon
Paul POULAIN