This patch removes support for the `x-koha-query` header entirely from
the codebase. For the purpose:
- $c->objects->search is adjusted
- Relevant tests are cleared/adapted
- The OpenAPI spec gets the header removed
To test:
1. Apply this patch and run:
$ ktd --shell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t \
t/Koha/REST/Plugin/Query.t \
t/db_dependent/api/v1/*.t
=> SUCCESS: Tests pass!
2. Sign off :-D
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds an exception to stash_embed that is thrown when we
find an x-koha-embed header that we're not expecting.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds optional support for using `:` as the modifier for
invoking the 'count' method for relations as discussed.
We retain support for `+` for backward compatability to prevent a
breaking change.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch updates the `av_expand` occurences to `strings` to match the
change of the return structure of `_strings`.
We replace `+av_expand` in the headers with `+strings`, the expected
object method name from `api_av_mapping` to `api_strings_mapping` and
the internal hash key from `av_expand` to `strings`.
Test plan
1) Run the included unit tests.. all should still pass.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Instead of a global av-expand flag (through a header) this patch
proposes to allow specifying +av_expand at the x-koha-embed header
level. This allows a more fine-grained control on what objects get avs
expanded. e.g.:
GET /patrons/123
x-koha-expand: +av_expand,checkouts.item+av_expand
=>
{
"_str": {
"city": { "str": "Córdoba", ... }
},
...
"checkouts": [
{
...,
"item": {
"_str": {
"not_for_loan_status": { "str": "Reference material", ... },
...
},
...
}
}
]
}
To test:
1. Run:
$ kshell
k$ prove t/db_dependent/Koha/Object.t \
t/Koha/REST/Plugin/Query.t \
t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
2. Sign off :-D
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Well spotted, this patch fixes the specs to be consistent and adds a
little detail to the POD as requested.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
With the introduction of proper validation of collection headers in
OpenAPI we no longer need to do our own validation here.
This patch removes the early validation of x-koha-embed headers in
preference to letting OpenAPI do it once and only once instead.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch updates the stash_embed method to use the parameters hash
enum instead of the top level x-koha-embed custom definition. This way
we can remove the requirement for defining this list twice.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
The maintainers of Mojolicious::Plugin::OpenAPI and friends fixed the
problem with using collectionFormat for Headers in the upstream library.
However, this highlighted an outstanding issue in our code when I ran
the unit tests. We have not been stripping x-koha-embed at the query
level as it did not exist in the reserved_words list. This patch
resolves that and gets the acquisitions_orders test passing again.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch adds 'x-koha-request-id' to the reserved param names list,
and makes the objects.search helper read it and send it back in the
response headers.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch improves the POD for the added helper, and also adds
references to the JSON::Validator issue that explains the current
implementation. A FIXME is added for when we upgrade our used
JSON::Validator version.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a simple helper that reads the x-koha-override request
header, and processes it to stash a hashref with the passed overrides.
No check on the overrides themselves is done, as they should be
validated using the OpenAPI spec.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/db_dependent/api/v1/auth_authenticate_api_request.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
While not that common nowadays, it is the syntax PHP uses, and
DataTables also generates such thing in 'traditional' mode. We should
support it as well.
This patch adds support for that. It does so by adding _order_by[] to
the reserved param names, and proper handling on the dbic_merge_sorting
helper.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
3- Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Bug 27680: (QA follow-up) Minor perlcritic issue
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch allows the preceeding test to pass. Strictly, we want a comma
delimited string for our _order_by parameter, but we cannot easily block
a traditional multi-passed parameter. As such the 'nice' thing to do is
handle it when such a thing is passed as it will pass through validation
regardless.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds proper handling for sorting a single column that is
constructed of multiple data entities.. i.e `"data": "string1:string2"`
It does NOT add support for filtering on multiple columns yet.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When a child of a prefetched element wasn't found on element's
prefetch_whitelist, no prefetch element was returned.. including element
it self. Now we return element's name if no child was found.
Also, embeded keys that ended with +count where taken literaly. Now we
search the correct key by stripping _count of embedded element if it was
declared as "is_count"
To test:
1. Apply test patch but not this one
2. prove t/Koha/REST/Plugin/Query.t
CHECK => tests shoul fail
3. Apply this patch
4. prove t/Koha/REST/Plugin/Query.t
SUCCESS => tests pass
5. Sign off
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
With this patch REST API request can order results by embedded columns.
Full path to the column must be given for it to work.
For example: If you are on biblio endpoint and you want to order by holding patron's card number you could
> GET /biblio/1?_order_by=item.holds.cardnumber HTTP/1.1
> x-koha-embed: item.holds
To test:
1. apply this patch
2. prove t/db_dependent/Koha/REST/Plugin/Objects.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds the possibility to object.search helper, to also filter by prefetched columns.
In order to dynamically add filter parameters, they must be coded as json and placed in the body of the request, coded as string in 'q' query parameter or as string in 'x-koha-query' header.
The coded json, is in fact dbix syntax.
To test:
1. apply this patch
2. prove t/Koha/REST/Plugin/Query.t t/db_dependent/Koha/REST/Plugin/Objects.t
3. Sign off
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 9978 should have fixed them all, but some were missing.
We want all the license statements part of Koha to be identical, and
using the GPLv3 statement.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes the Koha::Object(s) derived classes expose information
about prefetch-able relations. This is then used by a new helper to
generate the prefetch information for the DBIC query.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/Object* \
t/db_dependent/Koha/REST/Plugin/Objects.t \
t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds a + syntax to specify on the x-koha-embed header that we
want the count of a relation in the response. For example:
GET /patrons/1
x-koha-embed: checkouts+count
Would return a JSON representation of a Koha::Patron, with a new
attribute added: checkouts_count, which will be the result of calling
$patron->checkouts->count.
This is all done automatically in to_api. This patch makes parsing the
x-koha-embed header build the right structure for passing to the to_api
method.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Sponsored-by: ByWater Solutions
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch separates query parameters from path parameters, and uses exact matching for the later.
To test:
1. Apply this patch
2. prove t/Koha/REST/Plugin/Query.t t/db_dependent/Koha/REST/Plugin/Objects.t
SUCCESS => tests ok
3. Sign off
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Use the mapping instead of building a fake hash only to get the key name.
To test:
1. apply this patch
2. prove t/Koha/REST/Plugin/Query.t
3. sign off
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch introduces a helper for handling x-koha-embed headers on API
requests. It reads the embed definitions and adds them to the stash for
later use (either manually on the controllers, or in the objects.search
helper.
x-koha-embed needs to be defined as a list on the OpenAPI spec.
It throws an exception when invalid combinations are found.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass!
3.Sign off :-D
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes dbic_merge_sorting accept a result set as parameter and
solves a FIXME in _build_order_atom.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds a to_model parameter to dbic_merge_sorting so it is
passed when used (for example from objects.search). The to_model param
is passed along to the _build_order_atom method where it is finally
used.
In the process I wrote tests that reflected some problems in the current
code:
- Mojolicious automatically returns a scalar if a query parameter only
happens once on a request. The code expected an arrayref in every case.
- There's a design issue that forced me to use some hacky code in
_build_order_atom.
The first issue is dealth with, by using Scalar::Util::reftype as the
Perl docs recommend.
The second issue, I don't plan to clean it here, as there's ongoing work
on a Koha::Objects->search_from_api method that will obsolete this code
most probably (see bug 23893 for a better picture of where the mappings
will be living soon).
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass!!
3. Sign off :-D
Sponsored-by: ByWater Solutions
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This causes test failures in D8 and would require packaging our own perlcritic
Reverting until future date when consequences do not outweigh benefits
This reverts commit ecc94da6b2.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Test plan:
1) Check all Koha/REST files and confirm that 'use Modern::Perl;' has
been removed.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch creates the 'build_query_params' helper, instead of the
original function in Koha::Objects.
Unit tests are removed for Koha::Objects::_build_query_params_from_api and
written for the helper plugin.
The objects.search helper gets a call to build_query_params added. Tests for it
updated to match this behaviour change.
To test:
- Apply this patches
- Run:
$ kshell
k$ prove t/Koha/REST/Plugin/Query.t \
t/db_dependent/Koha/Objects.t \
t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
- Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch makes the helper handling _order_by params expect a list of
values instead of a (to-be-splitted) string.
The idea is that the OpenAPI plugin will take care of splitting
pipe-delimited values if the spec is correctly defined.
Note: In the process I noticed + on the URL represents a space, so the
helper function is updated to handle both + and %2B as ascending.
To test:
- Run:
$ kshell
k$ prove t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass!
- Sign off :-D
Edit: Removed rebasing leftover making the tests fail.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch introduces a helper function called 'dbic_merge_sorting'
to Koha::REST::Plugin::Query.
This simple function adds SQL::Abstract order_by attribute to the passed
$filter hashref, as explained in the POD.
It introduces a syntax for passing sorting params on the request to the REST api.
The proposed syntax has been found in the wild, and is pretty trivial to parse/work with:
GET /api/v1/<endpoint>?order_by=+column_1|-column_2|column_3
As explained on the POD, + stands for 'asc' and - for 'desc'. If ommited, it defaults to the
DB engine default (usually asc).
To test:
- Apply this patches
- Run:
$ sudo koha-shell kohadev
k$ cd kohaclone
k$ prove t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass! And they make sense! :-P
- Sign off :-D
Edit: renamed params to match DBIC terminology. My bad :-D (tcohen)
Sponsored-by: Camden County
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch introduces a Mojolicious plugin to be used on the REST api.
It adds a helper method:
generate_dbic_query
===================
When used, it generates what's needed to perform a search on
DBIC/Koha::Objects like this:
my $params = $c->validation->output;
my ($filtered_params, $reserved_params) = $c->extract_reserved_params($params);
my $filter = do_smth($filtered_params, $reserved_params);
my $attributes = do_smth_reserved($reserved_params);
my $patrons = Koha::Patrons->search( $filter, $attributes );
It introduces reserved param names:
- _match
- _order_by
- _page
- _per_page
They are reserved for later usage (pagination, matching algorithm on building DB queries)
All the plugin's behaviour is tested.
To test:
- Run:
$ sudo koha-shell kohadev
k$ cd kohaclone
k$ prove t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass!
- Sign off :-D
Sponsored-by: ByWater solutions
Sponsored-by: Camden County
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Tomas Cohen Arazi
Martin Renvoize
Jonathan Druart
Agustin Moyano
Nick Clemens
Jasmine Amohia