Koha-clms/t
Jonathan Druart bffa27725a Bug 18066: Hea V2 [squashed]
This patch is the Koha part of the Hea v2 project.
You can find the (testing) code for the server at
  hea-ws  - https://github.com/joubu/hea-ws/commits/v2
  hea-app - https://github.com/joubu/hea-app/commits/v2
They contain the different pull requests made over the last 6 months.

More information on Hea at https://wiki.koha-community.org/wiki/KohaUsageStat_RFC
The goal of this commit message is to provide an overview of what could
be a new version of Hea.

Prior to these changes, the Hea database was filled with 1 line per Koha
installation. System preferences were filled by the libraries and a
cronjob (share_usage_with_koha_community.pl) collected these values to send
them to a webservice (hea-ws/upload.pl).

With the need to collect more data we would want to collect data at the library
level (branch) and not at the installation level.
For instance the geolocation, the url or the country can be different from one
library to another, even if managed from the same Koha installation.
The Hea DB has been upgraded to reflect that change (see hea-app/sql/schema.sql).

The hidden goal of this patch is to make Hea sexier and explain
better to libraries how it can be useful to share their information
with the Koha community. I guess the main problem is the lack of
communication and explanations about what we are doing we these data.
To fill this gap I'd like to (TODO)
  1. Communicate on the ML about this new version of Hea (once it got
pushed and backported)
  2. Link the Privacy_Policy.md from the Hea interface
  3. Get help from a native English speaker to add
popup/help/info/whatever on "Home › Administration › Usage statistics",
to clearly explain what happens (and what will not happen!) when an option or
another is set.

You can find screenshot of this whole enhancement on bug 18066, comment 2.

What this patch does:
- Create a new branches.geolocation DB field
- Add 3 new sysprefs:
  * UsageStatsGeolocation
  * UsageStatsLibrariesInfo
  * UsageStatsPublicID
- Integrate the Leaflet JS library to get a fancy map to pick
geolocations

How does it works:
On the new administration page where statistics to share are configured,
there are several new things. It is now possible to share information either
per Koha installation or libraries. If UsageStatsLibrariesInfo is set,
the info at library level (url, name, country, geolocation) will be
sent to the Hea webservice. If it is not set, you can decide to fill
UsageStatsLibraryUrl, UsageStatsLibraryName, UsageStatsCountry,
UsageStatsGeolocation to share these information. Note that even if the
data are retrieved at installation level, it's better to fill the prefs
as well: On the Hea website the different libraries defined for a given
Koha installation could be displayed on the same page.
This page is a public page which will be attributed to every
installation (with the pref UsageStatsPublicID). On this page all the
info available publicly will be displayed.

TODO later:
- Add a button on the administration page to delete the info shared
publicly. It will be easy to show that the info are no longer displayed
on the public page.
- Add an icon per Koha installation to get a better "public page"
- Any suggestions?

Test plan:
We will need to test hea-ws, hea-app and the Koha-side code to test the
whole enhancement.
1/ To start, clone the hea-ws and hea-app project and checkout the
'master' branch (*not* 'v2')
2/ Create the hea database and user
  CREATE DATABASE hea
  CREATE USER 'hea'@'localhost' IDENTIFIED BY 'hea';
  GRANT ALL PRIVILEGES ON hea.* TO 'hea'@'localhost';
  FLUSH PRIVILEGES;
3/ Fill the DB with some data
  mysql hea < hea-app/sql/schema.sql
  mysql hea < hea-app/sql/sql/mock-data.sql
4/ Checkout the 'v2' branch for both hea-ws and hea-app
5/ Execute the upgrade DB script
  % cd hea-app
  % perl -p -i -e 's/REPLACE_ME/hea/' sql/upgrade.pl # Fill the DB info
  % perl sql/upgrade.pl
Now the DB is using the v2 structure. That means we have 1 installation
row per library previously defined. 1 library row has also been created.
5/ Configure hea-ws
% echo '192.168.50.1 hea.koha-community.org' >> /etc/hosts
<VirtualHost *:80>
  DocumentRoot "/path/to/hea-ws"
  ServerName "hea.koha-community.org"
  <Directory "/">
    Options +ExecCGI
    Require all granted
    AddHandler cgi-script .pl
  </Directory>
</VirtualHost>

And enable it with a2ensite, then restart apache.
The copy the database.yml.sample to database.yml and edit it to fill the
DB info.

6/ Launch the hea-app
  % cd hea-app
  % edit README.md # to install the missing modules
  % cp environments/config.yml environments/development.yml
  % edit environments/development.yml # to fill the DB info
  % perl bin/app.pl
Then hit localhost:3000
You should see a local version of Hea with sample data

7/ Back to Koha side
A. We will test that the webservice still works with previous version of Koha (without v2)
a. Do not configure Hea
  % perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
Then hit localhost:3000
=> Nothing added
b. Configure Hea on admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> New library added
c. Modify the Hea configuration
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Info are modified

B. Not we will test that it works with the new version (much more fun ;))
% git checkout hea-v2 # koha
a. Configure Hea using /admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Check the result on localhost:3000
b. Share libraries's info
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
c. Continue to play a bit and share the info.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Mason James <mtj@kohaaloha.com>
2017-05-24 13:44:59 +12:00
..
Acquisition Bug 14778: Remove t/Acquisition/Invoices.t 2015-10-23 12:01:19 -03:00
Biblio Revert "Bug 17641: Fix t/Biblio/Isbd.t" 2016-12-31 14:44:43 +13:00
Budgets
…
Circulation Bug 14632: Add Copyright for the Koha Dev Team 2016-04-29 18:06:40 +00:00
DataTables Bug 15252 - Patron search on start with does not work with several terms 2015-12-30 13:05:54 +00:00
External Bug 16636 [QA Followup] - Move tests to t/External 2016-06-15 09:08:38 +02:00
Koha Bug 12748: (QA followup) make new tests pass 2016-04-26 20:20:14 +00:00
Members Bug 15690: Hardcoded 16 is uncool 2016-12-12 14:54:10 +13:00
Number Bug 15084: Replace C4::Budgets::GetCurrencies with Koha::Acquisition::Currencies->search 2016-03-03 20:39:01 +00:00
SIP Bug 15006: Centralize timeout logic and allow zero client timeout 2016-08-01 11:52:51 +02:00
Search Bug 10807: Add an authority search history for the OPAC 2014-05-05 02:32:27 +00:00
Serials
…
db_dependent Bug 18066: Hea V2 [squashed] 2017-05-24 13:44:59 +12:00
edi_testfiles Bug 7736: Support Ordering via Edifact EDI messages 2016-04-01 20:03:17 +00:00
lib Bug 17726: TestBuilder - Add default values 2017-02-22 21:58:03 +13:00
00-checkdatabase-version.t Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
00-deprecated.t
…
00-load.t Bug 17372: Standardize Elasticsearch paths 2016-10-22 13:56:11 +02:00
00-merge-conflict-markers.t Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
00-testcritic.t Bug 15258: Prevent unused declared variables 2015-12-30 17:24:30 -07:00
00-valid-xml.t Bug 16608 - Add xsd extension to t/00-valid-xml.t 2016-06-15 09:02:59 +02:00
AuthUtils.t
…
Auth_with_shibboleth.t Bug 17681: Exiisting typos might thow some feees when recieved 2017-01-03 14:44:41 +13:00
AuthoritiesMarc_MARC21.t
…
AuthoritiesMarc_UNIMARC.t
…
Barcodes_EAN13.t
…
Barcodes_annual.t
…
Barcodes_hbyymmincr.t
…
Barcodes_incremental.t
…
Biblio.t Bug 16169: Change prototype for C4::Biblio::TransformMarcToKoha 2016-04-07 00:04:21 +00:00
Biblio2.t Bug 16556 - KohaToMarcMapped columns sharing same field with biblio(item)number are removed. 2016-09-15 17:14:21 +02:00
Bookseller.t
…
Boolean.t Bug 13814: (QA followup) test for generated warnings 2015-03-15 09:03:33 -03:00
Branch.t
…
Breeding.t Bug 13279: t/Breeding.t shouldn't depend on the DB 2014-11-20 09:39:53 -03:00
Budgets.t
…
Cache.t Bug 16166: Fix t/Cache.t tests 2016-08-01 11:46:24 +02:00
Calendar.t Bug 16539: Add tests 2016-05-23 16:48:57 +00:00
Charset.t Bug 14112: Silence warnings in t/Charset.t 2015-05-27 14:32:41 -03:00
Circulation_barcodedecode.t Bug 15151: Avoid DB access to load C4::Members 2015-11-08 13:10:13 -03:00
ClassSortRoutine.t
…
ClassSortRoutine_Dewey.t Bug 8837 [QA Followup] - Add unit tests 2014-10-28 09:39:44 -03:00
ClassSortRoutine_Generic.t
…
ClassSortRoutine_LCC.t
…
ClassSource.t
…
Context.t Bug 14751: [QA Follow-up] Unit tests for interface method 2016-03-02 04:21:26 +00:00
Contract.t
…
Creators.t Bug 16622: some tests triggered by prove t fail for unset KOHA_CONF 2016-08-07 09:43:41 +02:00
DateUtils.t Bug 17476: Add a way to bypass dt_from_string processing time for slow servers 2016-12-12 14:54:10 +13:00
Debug.t Bug 14114: Silence warns and cleanup t/Debug.t 2015-05-22 09:33:34 -03:00
EdiInvoice.t Bug 7736: Support Ordering via Edifact EDI messages 2016-04-01 20:03:17 +00:00
Edifact.t Bug 7736: Support Ordering via Edifact EDI messages 2016-04-01 20:03:17 +00:00
Ediorder.t Bug 7736: Support Ordering via Edifact EDI messages 2016-04-01 20:03:17 +00:00
Ediordrsp.t Bug 7736: Support Ordering via Edifact EDI messages 2016-04-01 20:03:17 +00:00
Form_MessagingPreferences.t
…
Heading.t
…
Images.t Bug 15150: Make t/ tests skip if Test::DBIx::Class absent 2015-11-06 12:25:27 -03:00
ImportBatch.t Bug 13281: t/ImportBatch.t shouldn't depend on the DB 2014-11-20 09:53:36 -03:00
Installer_PerlDependencies.t
…
Installer_PerlModules.t Bug 17990: Refactor Perl module versions check 2017-01-30 00:56:20 +13:00
Installer_pm.t Bug 17990: Refactor Perl module versions check 2017-01-30 00:56:20 +13:00
ItemCirculationAlertPreference.t
…
Koha.t Bug 15800: Koha::AuthorisedValues - Remove C4::Koha::IsAuthorisedValueCategory 2016-03-02 03:54:16 +00:00
Koha_Email.t Bug 13172 t/Koha_Email.t was database dependent 2014-10-31 12:10:32 -03:00
Koha_MetadataRecord.t Bug 8064: Fix unit tests for createMergeHash 2015-11-09 15:08:57 -03:00
Koha_Template_Plugin_Cache.t
…
Koha_Template_Plugin_Koha.t Bug 13758: (QA followup) revert case change that broke the tests 2015-05-07 14:12:14 -03:00
Koha_Util_FrameworkPlugin.t Bug 13437: Preliminary changes for marc21 plugins field 008 2015-06-10 12:51:26 -03:00
Koha_Util_MARC.t Bug 8064: Fix unit tests for createMergeHash 2015-11-09 15:08:57 -03:00
Labels.t Bug 9978: (followup) Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:43 -03:00
Labels_split_ccn.t Bug 9978: (followup) Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:43 -03:00
Labels_split_ddcn.t Bug 9978: (followup) Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:43 -03:00
Labels_split_lccn.t Bug 9978: (followup) Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:43 -03:00
Languages.t Bug 16675: fix breakage of t/Languages.t 2016-06-15 08:25:42 +02:00
Letters.t Bug 15150: Make t/ tests skip if Test::DBIx::Class absent 2015-11-06 12:25:27 -03:00
Log.t
…
Logger.t Bug 14167: (QA followup) use warn instead of just STDERR 2015-07-21 10:50:17 -03:00
Matcher.t Bug 17304: Introduce C4::Matcher::_get_match_keys unit tests 2016-09-30 11:21:21 +02:00
Members_AttributeTypes.t Bug 12267: Remove borrower_attributes.password 2016-04-22 23:08:32 +00:00
Members_Attributes.t Bug 12267: Remove borrower_attributes.password 2016-04-22 23:08:32 +00:00
Members_Messaging.t
…
Message.t Bug 13282: t/Message.t shouldn't depend on the DB 2014-11-20 09:38:02 -03:00
NorwegianPatronDB.t Bug 11401: (followup) make tests run on absent deps 2014-11-14 15:32:25 -03:00
OpenLibrarySearch.t Bug 16649: Make OpenLibrarySearch test pass even if launches offline 2016-06-15 08:26:08 +02:00
Output.t Bug 11944: use CGI( -utf8 ) everywhere 2015-01-13 13:07:21 -03:00
Output_JSONStream.t
…
Overdues.t Bug 13283: t/Overdues.t shouldn't depend on the DB 2014-11-20 09:37:39 -03:00
Patron.t Bug 15548: Move new patron related code to Patron* 2016-03-03 14:38:26 -07:00
Patroncards.t
…
Patroncards_Batch.t
…
Patroncards_Layout.t
…
Patroncards_Lib.t
…
Patroncards_Patroncard.t
…
Patroncards_Profile.t
…
Patroncards_Template.t
…
Prices.t Bug 16582 t/Price.t test should pass if Test::DBIx::Class is not available 2016-06-15 07:36:15 +02:00
Print.t
…
QueryParser.t Bug 12647: PQF QueryParser driver and unit tests fixes 2014-08-11 10:09:38 -03:00
RecordProcessor.t Bug 15871: Improve PerlCritic level for t/RecordProcessor.t 2016-03-03 22:02:50 +00:00
Review.t
…
Ris.t Bug 16668: Fix t/Ris.t tests 2016-06-15 08:25:23 +02:00
RotatingCollections.t Bug 13284: t/RotatingCollections.t shouldn't depend on the DB 2014-11-20 09:37:20 -03:00
SMS.t Bug 12642: t/SMS.t should not be db dependent 2014-07-29 11:27:55 -03:00
Scheduler.t
…
Scrubber.t Bug 14116: Silence noisy output for t/Scrubber. 2015-06-01 14:10:56 -03:00
Search.t Bug 13278: (QA followup) use t::lib::Mocks 2014-11-19 18:11:58 -03:00
Search_PazPar2.t Bug 14117: Silence warnings t/SearchPazPar2.t 2015-06-01 14:10:14 -03:00
SimpleMARC.t Bug 14098: Implement the copy_and_replace action for MTT 2015-09-07 11:17:13 -03:00
SocialData.t Bug 15150: Make t/ tests skip if Test::DBIx::Class absent 2015-11-06 12:25:27 -03:00
Stats.t
…
SuggestionEngine.t
…
SuggestionEngine_AuthorityFile.t Bug 13277: (QA followup) use t::lib::Mocks 2014-11-19 18:12:03 -03:00
TmplToken.t
…
Token.t t/Token.t merge typo fix 2017-05-03 13:15:43 +12:00
XSLT.t Bug 18089: Remove warnings from tests using DBIx::Class fixtures 2017-02-23 09:36:22 +13:00
dummy.t
…
perlcriticrc Bug 15258: Prevent unused declared variables 2015-12-30 17:24:30 -07:00
smolder_smoke_signal
…