Koha-clms/C4
Jonathan Druart 6c96189c0d Bug 18066: Hea V2
This patch is the Koha part of the Hea v2 project.
You can find the (testing) code for the server at
  hea-ws  - https://github.com/joubu/hea-ws/commits/v2
  hea-app - https://github.com/joubu/hea-app/commits/v2
They contain the different pull requests made over the last 6 months.

More information on Hea at https://wiki.koha-community.org/wiki/KohaUsageStat_RFC
The goal of this commit message is to provide an overview of what could
be a new version of Hea.

Prior to these changes, the Hea database was filled with 1 line per Koha
installation. System preferences were filled by the libraries and a
cronjob (share_usage_with_koha_community.pl) collected these values to send
them to a webservice (hea-ws/upload.pl).

With the need to collect more data we would want to collect data at the library
level (branch) and not at the installation level.
For instance the geolocation, the url or the country can be different from one
library to another, even if managed from the same Koha installation.
The Hea DB has been upgraded to reflect that change (see hea-app/sql/schema.sql).

The hidden goal of this patch is to make Hea sexier and explain
better to libraries how it can be useful to share their information
with the Koha community. I guess the main problem is the lack of
communication and explanations about what we are doing we these data.
To fill this gap I'd like to (TODO)
  1. Communicate on the ML about this new version of Hea (once it got
pushed and backported)
  2. Link the Privacy_Policy.md from the Hea interface
  3. Get help from a native English speaker to add
popup/help/info/whatever on "Home › Administration › Usage statistics",
to clearly explain what happens (and what will not happen!) when an option or
another is set.

You can find screenshot of this whole enhancement on bug 18066, comment 2.

What this patch does:
- Create a new branches.geolocation DB field
- Add 3 new sysprefs:
  * UsageStatsGeolocation
  * UsageStatsLibrariesInfo
  * UsageStatsPublicID
- Integrate the Leaflet JS library to get a fancy map to pick
geolocations

How does it works:
On the new administration page where statistics to share are configured,
there are several new things. It is now possible to share information either
per Koha installation or libraries. If UsageStatsLibrariesInfo is set,
the info at library level (url, name, country, geolocation) will be
sent to the Hea webservice. If it is not set, you can decide to fill
UsageStatsLibraryUrl, UsageStatsLibraryName, UsageStatsCountry,
UsageStatsGeolocation to share these information. Note that even if the
data are retrieved at installation level, it's better to fill the prefs
as well: On the Hea website the different libraries defined for a given
Koha installation could be displayed on the same page.
This page is a public page which will be attributed to every
installation (with the pref UsageStatsPublicID). On this page all the
info available publicly will be displayed.

TODO later:
- Add a button on the administration page to delete the info shared
publicly. It will be easy to show that the info are no longer displayed
on the public page.
- Add an icon per Koha installation to get a better "public page"
- Any suggestions?

Test plan:
We will need to test hea-ws, hea-app and the Koha-side code to test the
whole enhancement.
1/ To start, clone the hea-ws and hea-app project and checkout the
'master' branch (*not* 'v2')
2/ Create the hea database and user
  CREATE DATABASE hea
  CREATE USER 'hea'@'localhost' IDENTIFIED BY 'hea';
  GRANT ALL PRIVILEGES ON hea.* TO 'hea'@'localhost';
  FLUSH PRIVILEGES;
3/ Fill the DB with some data
  mysql hea < hea-app/sql/schema.sql
  mysql hea < hea-app/sql/sql/mock-data.sql
4/ Checkout the 'v2' branch for both hea-ws and hea-app
5/ Execute the upgrade DB script
  % cd hea-app
  % perl -p -i -e 's/REPLACE_ME/hea/' sql/upgrade.pl # Fill the DB info
  % perl sql/upgrade.pl
Now the DB is using the v2 structure. That means we have 1 installation
row per library previously defined. 1 library row has also been created.
5/ Configure hea-ws
% echo '192.168.50.1 hea.koha-community.org' >> /etc/hosts
<VirtualHost *:80>
  DocumentRoot "/path/to/hea-ws"
  ServerName "hea.koha-community.org"
  <Directory "/">
    Options +ExecCGI
    Require all granted
    AddHandler cgi-script .pl
  </Directory>
</VirtualHost>

And enable it with a2ensite, then restart apache.
The copy the database.yml.sample to database.yml and edit it to fill the
DB info.

6/ Launch the hea-app
  % cd hea-app
  % edit README.md # to install the missing modules
  % cp environments/config.yml environments/development.yml
  % edit environments/development.yml # to fill the DB info
  % perl bin/app.pl
Then hit localhost:3000
You should see a local version of Hea with sample data

7/ Back to Koha side
A. We will test that the webservice still works with previous version of Koha (without v2)
a. Do not configure Hea
  % perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
Then hit localhost:3000
=> Nothing added
b. Configure Hea on admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> New library added
c. Modify the Hea configuration
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Info are modified

B. Not we will test that it works with the new version (much more fun ;))
% git checkout hea-v2 # koha
a. Configure Hea using /admin/usage_statistics.pl
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
=> Check the result on localhost:3000
b. Share libraries's info
perl misc/cronjobs/share_usage_with_koha_community.pl -f -v
c. Continue to play a bit and share the info.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
(cherry picked from commit 63a1adff90)
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
2017-05-10 16:13:56 +02:00
..
AuthoritiesMarc Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Barcodes Bug 16635: [QA Follow-up] Eliminate some global package vars 2016-06-16 12:03:37 +02:00
Bookseller Bug 10402 follow-up: choose contacts for claims 2014-08-26 11:45:59 -03:00
ClassSortRoutine Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Creators Bug 18044: Label Batches not displaying 2017-02-09 10:31:17 +01:00
External Bug 16636: [QA Follow-up] Make BakerTaylor plack safe 2016-06-16 12:05:38 +02:00
Form Bug 16259: Replace CGI->param with CGI->multi_param in list context - part 2 2016-05-17 07:44:16 +02:00
Heading Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
ILSDI Bug 16864: Silence warnings in t/db_dependent/ILSDI_Services.t 2016-08-17 15:42:49 +02:00
Installer 3.22.10 debian control fixes for debian package 2016-09-16 12:14:29 +02:00
Labels Bug 14781: Creation of barcode types 2of5 not functional 2015-11-05 10:31:46 -03:00
Linker Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Members Bug 16443: Make C4::Members::Statistics plack safe 2016-06-16 10:31:37 +02:00
OAI Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Output Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Patroncards Bug 18244: Patron card creator does not take in account fields with underscore (B_address etc.) 2017-04-21 12:09:12 +02:00
Reports Bug 14982: Remove C4::Dates from guided report wizard and dictionary 2015-10-26 11:20:11 -03:00
SIP Bug 12021 - SIP2 checkin should alert on transfer and use CT for return branch 2017-05-10 11:46:33 +02:00
Search Bug 17323: Column search_history.time cannot be null 2016-11-30 14:06:47 +01:00
Serials Bug 14383: C4: Fix some typos (mostly in comments and documentation) 2015-06-22 17:34:45 -03:00
Utils Bug 18094: Only search in searchable patron attributes if searching in standard fields 2017-03-31 15:16:57 +02:00
Accounts.pm Bug 18266: Fix internal error when paying fine for lost item without.. item 2017-04-21 12:44:08 +02:00
Acquisition.pm Bug 14535: Late orders does not show orders with price = 0 2017-04-21 12:06:30 +02:00
Auth.pm Bug 18443: Get rid of warning 'uninitialized value $user' in C4/Auth.pm 2017-05-10 12:05:28 +02:00
Auth_cas_servers.yaml.sample Bug 17065 - Rename C4/Auth_cas_servers.yaml.orig 2016-08-17 16:00:36 +02:00
Auth_with_cas.pm Bug 17481: Fix incorrect merge of bug 11048 (logout redirection for CAS authentication) 2016-11-30 13:08:58 +01:00
Auth_with_ldap.pm Bug 17775 - Add new user with LDAP not works under Plack 2017-02-09 10:28:32 +01:00
Auth_with_shibboleth.pm Bug 16818: External auth redirect broken under Plack 2016-08-17 15:35:24 +02:00
AuthoritiesMarc.pm Bug 17913: [3.22.x] Authority merge fix 2017-02-20 11:47:33 +01:00
BackgroundJob.pm Bug 13606 Clear job data in session when the job is completed 2015-04-23 13:18:48 -03:00
Barcodes.pm Bug 13813: Remove deprecated module C4::Dates from system 2015-09-18 12:40:55 -03:00
Biblio.pm Bug 16556 - KohaToMarcMapped columns sharing same field with biblio(item)number are removed. 2016-09-16 12:01:31 +02:00
Bookseller.pm Bug 14535: Update the supplier filter too 2017-04-21 12:06:30 +02:00
Boolean.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Branch.pm Bug 15628: Remove get_branchinfos_of vestiges 2016-02-04 22:29:05 +01:00
Breeding.pm Bug 13987: Fix server name in z39.50 authority search results 2015-04-22 14:39:00 -03:00
Budgets.pm Bug 17872: Fix small error in GetBudgetHierarchy and one of its calls 2017-04-21 13:09:57 +02:00
Calendar.pm Bug 15333: Use Koha::Cache to cache exception_holidays instead of a package variable 2016-06-09 09:32:46 +02:00
Category.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Charset.pm Bug 14078: (followup) converting from ISO5426 is not complete 2015-11-16 12:48:44 -03:00
Circulation.pm Bug 18364: [Follow-up] Also add an environment variable to prevent locking 2017-05-10 11:24:22 +02:00
ClassSortRoutine.pm Bug 14383: C4: Fix some typos (mostly in comments and documentation) 2015-06-22 17:34:45 -03:00
ClassSource.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Context.pm Bug 16813: Allow empty string for OPACBaseURL 2016-08-17 12:45:46 +02:00
Contract.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
CourseReserves.pm Bug 15530 - Editing a course item via a disabled course disables it even if it is on other enabled courses 2016-01-27 15:33:36 +01:00
Creators.pm Bug 14059: Delete all export of get_column_names 2015-05-22 17:02:21 -03:00
Csv.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Debug.pm Bug 14870: (followup) Remove mention of C4::Dates from C4/Debug.pm 2015-11-19 13:05:06 -03:00
HTML5Media.pm Bug 12885: Fix if url contains +*... and HTML5 Media is enabled 2015-09-02 14:38:29 -03:00
Heading.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
HoldsQueue.pm Bug 15062: Holds queue with Transport Cost Matrix will transfer item even if transfers disabled 2015-11-23 11:32:53 -03:00
Images.pm Bug 14778: Install fixtures for t/Images.t 2015-10-23 12:01:18 -03:00
ImportBatch.pm Bug 6852: Staged import reports wrong success for items 2016-09-16 11:09:21 +02:00
ImportExportFramework.pm Bug 17389: Clear logs 2016-10-24 12:02:11 +02:00
InstallAuth.pm Bug 12930: Web installer does not show login errors 2017-04-21 13:12:23 +02:00
Installer.pm Bug 17234: Need to separate KEY and FOREIGN KEY checks 2017-01-19 16:36:53 +01:00
ItemCirculationAlertPreference.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
ItemType.pm Bug 16356: Fix Error 500 when returning an item which itemtype is not in itemtypes table 2016-04-28 12:51:07 +02:00
Items.pm Bug 17512: Improve handling dates in C4::Items 2017-02-20 10:23:38 +01:00
Koha.pm Bug 16716: Replaced wrong GROUP BY with DISTINCT 2016-08-17 13:40:41 +02:00
Labels.pm Bug 8315 - remove use C4::* version 2012-07-13 14:17:20 +02:00
Languages.pm Bug 16770: Remove wrong caching of 3 subroutines in C4::Lancuages 2016-08-16 15:33:10 +02:00
Letters.pm Bug 17903: Fix possible SQL injection in serial claims 2017-01-30 15:54:18 +01:00
Linker.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Log.pm Bug 15939: Action logs - Do not default dates to today 2016-03-07 11:35:14 +01:00
MarcModificationTemplates.pm Bug 14098: Implement the copy_and_replace action for MTT 2015-09-07 11:17:13 -03:00
Matcher.pm Bug 15745: C4::Matcher gets CCL parsing error if term contains ? (question mark) 2016-04-06 16:38:52 +02:00
Members.pm Bug 17782: deletedborrowers.updated_on should be set to current timestamp 2017-02-09 10:51:54 +01:00
Message.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
NewsChannels.pm Bug 16550: Can't set opac news expiration date to NULL, it reverts to today 2016-05-25 11:27:22 +02:00
Output.pm Bug 15111: Change X-Frame-Options with SAMEORIGIN 2016-04-22 10:51:11 +02:00
Overdues.pm Bug 14390: [Follow-up] Only update FU record in UpdateFine 2016-09-16 08:06:40 +02:00
Patroncards.pm Bug 8315 - remove use C4::* version 2012-07-13 14:17:20 +02:00
Print.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Ratings.pm Bug 14383: Fix POD error in C4/Ratings.pm 2015-06-22 17:34:47 -03:00
Record.pm Bug 12357: Following whitespace failures on koha qa test tools. 2015-09-07 13:59:07 -03:00
Reports.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Reserves.pm Bug 17995 - HOLDPLACED notice should have access to the reserves table 2017-04-21 11:38:31 +02:00
Review.pm Bug 13969: Replace calls to $sth->fetchrow* with a call to $dbh->selectrow* and Cleaning files 2015-05-14 11:11:41 -03:00
Ris.pm Bug 16245: [QA Follow-up] Replace typeofrecord2 by biblevel 2016-10-24 12:48:33 +02:00
RotatingCollections.pm Bug 15066: Make transfer rotating collection works under Plack 2015-11-05 09:50:09 -03:00
SMS.pm Bug 14421: Corrected example in SMS.pm to working version with hashref. 2015-06-22 11:08:29 -03:00
Scheduler.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Scrubber.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Search.pm Bug 17821 - due date in intranet search results should use TT date plugin 2017-05-10 10:21:46 +02:00
Serials.pm Bug 7728: [QA Follow-up] Fix POD whitespace 2017-04-21 12:14:54 +02:00
Service.pm Bug 11559: Supporting changes for Rancor 2015-10-27 12:16:05 -03:00
ShelfBrowser.pm Bug 16315 - OPAC Shelfbrowser doesn't display the full title 2016-05-16 10:08:03 +02:00
SocialData.pm Bug 9978: (followup) Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:43 -03:00
Stats.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Suggestions.pm Bug 16325: Do not return all suggestions if search for STATUS='' 2016-05-16 13:24:14 +02:00
TTParser.pm Bug 14383: C4: Fix some typos (mostly in comments and documentation) 2015-06-22 17:34:45 -03:00
Tags.pm Bug 16637: Dependency for C4::Tags not listed 2016-06-16 12:07:29 +02:00
Templates.pm Bug 18010: Remove potential exposure from gettemplate 2017-03-31 15:20:04 +02:00
TmplToken.pm Bug 6679: (follow-up) fix 9 perlcritic violations in C4/TmplTokenType.pm 2016-01-27 15:26:02 +01:00
TmplTokenType.pm Bug 6679: (follow-up) fix 9 perlcritic violations in C4/TmplTokenType.pm 2016-01-27 15:26:02 +01:00
UsageStats.pm Bug 18066: Hea V2 2017-05-10 16:13:56 +02:00
XISBN.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
XSLT.pm Bug 7441 - search results showing wrong branch? 2016-08-23 16:13:23 +02:00