This test ensures all Text::CSV usage in Koha includes proper formula
injection protection to prevent CSV formula injection attacks.
The test scans all Perl modules and verifies that Text::CSV constructors
include the 'formula' parameter set to a safe mode ('empty', 'die',
'croak', 'diag', or numeric 1-5).
To test:
1. Apply this patch
2. Run:
$ ktd --shell
k$ prove xt/author/Text_CSV_Various.t
=> FAIL: Test should fail showing files without formula protection:
- Koha/Patrons/Import.pm
- Koha/BackgroundJob/ImportKBARTFile.pm
- Koha/ERM/EUsage/CounterFile.pm
3. Apply the follow-up patch to fix the violations
4. Repeat 2
=> SUCCESS: All tests should pass
5. Sign off :-D
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
We have insert_final_newline = true to ensure files will contain a new
line at the end. However we have seen commits with "No new line" added
in the git log.
We should have a xt test to prevent more occurrences.
We should also have a test at the QA script level:
https://gitlab.com/koha-community/qa-test-tools/-/issues/96
Test plan:
prove xt/find-missing-new-lines-at-the-end-of-file.t
should return green.
Remove the new line at the end of a file, eg.
vim -b mainpage.pl
:set noeol
:wq
Now
prove xt/find-missing-new-lines-at-the-end-of-file.t
is failing on mainpage.pl.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Including chmod 755 xt/use-test-no-warnings.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Maybe we could remove the warnings from misc/devel/tidy.pl if the file
was originally empty but for now it feels safer to skip the tidy of
empty files
Test plan:
prove xt/tt_tidy.t xt/use-test-no-warnings.t
should return green
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
We've got those tests running without warnings now too.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds a new xt test (xt/use-test-no-warnings.t) to make sure
all our Perl test files will use Test::NoWarnings.
It also adds Test::NoWarnings to some files that were missing it and
that didn't need any other changes.
However we still have 7 non-trivial tests that we will consider
exception for now, until they are fixed.
Test plan:
prove xt/use-test-no-warnings.t
should return green.
And all other tests modified by this patch (the qa script will do that
for you)
You can also test the 2 selenium tests:
1. DROP and CREATE the DB
DROP DATABASE koha_kohadev;
CREATE DATABASE koha_kohadev;
restart_all
2. KOHA_TESTING=1 prove t/db_dependent/selenium/00-onboarding.t
3. Repeat 1
2. KOHA_TESTING=1 prove t/db_dependent/selenium/01-installation.t
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The test added in xt/api.t was not fully accurate, we skipped the
columns that have the same name as the api attribute (ie. the ones not
part of to_api_mapping)
Test plan:
prove xt/api.t should pass
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch makes a couple changes to the tests:
* Remove check for Copyright line. The Copyright line should not be mandatory.
* Remove dependency of GPL-related checks on the copyrigth line
existence. (this highlights several missed checks).
* Skip testing-related templates. Koha ships some templates that are not
excluded correctly.
This 'fix' will highlight existing errors that were skipped until now.
To test:
1. Run:
$ ktd --shell
k$ prove xt/find-license-problems.t
=> FAIL: Tests pass! Some files should make them fail!
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests fail!
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
And only those files.
Currently the test is failing with
02:52:31 koha_1 | # Failed test at xt/find-license-problems.t line 65.
02:52:31 koha_1 | # got: '0'
02:52:31 koha_1 | # expected: '1'
02:52:33 koha_1 | # File ./misc/translator/po/LICENSE has wrong copyright: hasgpl=1, hasv3=1, hasorlater=1, haslinktolicense=0, hasfranklinst=0
02:52:33 koha_1 | # Looks like you failed 1 test of 1381.
02:52:33 koha_1 | [00:51:24] xt/find-license-problems.t
Because a LICENSE file has been added to koha-l10n, and now this test fails on it.
27ab9ff4d2
But we should not test files that are not part of the git index.
Test plan:
0. Do not apply this patch
1. Edit xt/find-license-problems.t, add
warn scalar(@files);
2. Run `prove xt/find-license-problems.t`
=> Notice that 4056 files are processed
Note that you need to have a clean git repo. You can run the following
git command BUT BEWARE /*\ it will remove all changes not committed
`git clean -d -f`
3. Apply this patch, prove again
=> Tests are passing, with the same number of tests
xt/find-license-problems.t .. ok
All tests successful.
Files=1, Tests=4056
Note that I think we lost one file, as Test::NoWarnings is adding a
test. But I don't think it is worth the trouble to find out which one is
missing.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds a new subtest to xt/api.t
It ensures that maxLength will appear in the rest api spec file when
there is a size constraint at the DB level
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
We are ready to reuse Koha::Devel::Files in other places where
`git ls-files` is used.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch changes the regex so it doesn't expect the exclusion
parameters to only be files but accept directories too.
The risk of this is we might at some point catch more than we want.
I don't think it is worth spending more time in it, though.
To test:
1. Run:
$ ktd --shell
k$ prove xt/author/valid-templates.t
=> SUCCESS: Tests pass!
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests pass! We don't need to list each individual file
anymore and it will still work!
4. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
bullseye has 2.0.0-1
bookworm has 2.2.2-1
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
TODO: We will need to adjust this path with the db_rev one
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Adjusted path.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We certainly want to improve this test and run it in parallel.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We exclude (for now) some files that needs additional work:
1. Koha/Account/Credit.pm, Koha/Account/Debit.pm and Koha/Old/Hold.pm
"Inconsistent hierarchy during C3 merge of class"
2. misc/translator/TmplTokenizer.pm
Can't locate VerboseWarnings.pm in @INC
We also ignore some known warnings.
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Jenkins is not failing when git command fails.
We edited the config and it broke the git repo:
13:32:35 koha_1 | error: unable to normalize alternate object path: /srv/jenkins/workspace/Koha_Source/.git/objects
13:32:35 koha_1 | fatal: not a tree object
13:32:35 koha_1 | [12:29:31] xt/check_makefile.t
So tests were not running correctly
Maybe it will be a problem for people running the tests without being
inside a git repo, but for now we need to ensure the community CI does
what we think it does.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
It was not failing when warnings were found
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
podchecker is used by the QA script and catch more than Test::Pod
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
It only tested files within C4 and Koha directories.
Now we test all the perl files that are in the git index.
Test plan:
0. Do not apply those patches
1. prove xt/author/podcorrectness.t
Notice "Tests=1208" meaning that 1208 files are tested
2. Apply this patch (not the second one)
It fails, and more files are being tested (3254)
xt/author/podcorrectness.t .. 1994/3254
xt/author/podcorrectness.t .. 2726/3254 # Looks like you failed 1 test of 3254.
3. Apply the second patch
=> It passes!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Now perltidy.t will check 2958 files instead of 2954. Which matches
with the 4 .PL files.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
They were still using prettier directly. They should use the tidy.pl as
it is doing some wrapper work.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This commit is generated using:
% perl misc/devel/tidy.pl
*within* ktd, to get the same version of perltidy than what will be used
by our CI (currently v20230309).
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We want to test the files from the git index only
On my laptop:
real 3m54.692s
user 3m51.466s
sys 0m0.994s
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The test is not catching missing op in forms that are not the first POST
form of the template.
Found when fixing the test for bug 37309. It was not catching the
missing op in the second form.
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
to test...
1/ set koha repo to state where yarn.lock is not updated
$ git reset --hard 67db70d4
2/ run test, observe FAIL
$ prove ./xt/verify-yarnlock.t
./xt/verify-yarnlock.t .. error Your lockfile needs to be updated, but yarn was run with `--frozen-lockfile`.
./xt/verify-yarnlock.t .. 1/1
# Failed test 'verify yarn.lock file is updated correctly'
...
Result: FAIL
3/ set koha repo to state where yarn.lock is updated
$ yarn install
4/ note yarn.lock is now updated
$ git status
...
modified: yarn.lock
5/ run test, observe SUCCESS
$ prove -v ./xt/verify-yarnlock.t
./xt/verify-yarnlock.t ..
ok 1 - verify yarn.lock file is updated correctly
All tests successful.
Files=1, Tests=1, 1 wallclock secs ( 0.02 usr 0.01 sys + 1.16 cusr 0.27 csys = 1.46 CPU)
Result: PASS
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a test for well defined 400 responses on all verbs and
paths on the API spec.
The tests verify:
* Presence of 400 response definition
* The description must start with 'Bad request' (needs coding guideline)
* If DBIC queries are allowed on the route, then `invalid_query` needs
to be mentioned in the description.
All routes get fixed to make the tests pass.
To test:
1. Apply this patch
2. Run:
$ ktd --shell
k$ yarn api:bundle
k$ prove xt/api.t
=> SUCCESS: Tests pass!
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Tomas Cohen Arazi
Jonathan Druart
Marcel de Rooy
Martin Renvoize
Katrin Fischer
Victor Grousset/tuxayo
Koha Development Team
Mason James