README.md

The COIN project
================

`Coin` is Illyse's Information System, designed to manage both members
and Internet accesses, such as through DSL, VPN, wireless…


![example of a coin user view : the subscriptions list](doc/_img/user-subscriptions.png)

It is written in Django, and features a generic configuration interface,
which allows to implement custom backends for different technologies.
Currently implemented is a LDAP-based backend for OpenVPN, and a very
simple DSL backend, without any authentication (useful for "white label"
DSL).

Coin currently only works with python2, because `python-ldap` is (as of
2013) not compatible with python3.

The project page (issue, wiki, etc) is here:

  https://code.ffdn.org/FFDN/coin/

A mailing list is available, for both users and developers: https://listes.illyse.org/wws/info/si

License
=======

Coin is released under the GNU Affero GPL 3.0 license.  See `LICENSE` for details.

Extending Coin
==============

If you want to write your own backend, see `EXTENDING.md`.


Quickstart
==========

Virtualenv
----------

Using a virtualenv is recommended.  On Debian, install `python-virtualenv`.
On Archlinux, the package is called `python2-virtualenv`, and you must
replace the `virtualenv` command with `virtualenv2` in the following.

To create the virtualenv (the first time):

    virtualenv ./venv


To activate the virtualenv (you need to do this each time you work on
the project):

    source ./venv/bin/activate


Install dependencies. On Debian, you will probably need the
`python-dev`, `python-pip`, `libldap-dev`, `libpq-dev`, `libsasl2-dev`,
and `libjpeg-dev` packages.

    sudo apt-get install python-dev python-pip libldap2-dev libpq-dev libsasl2-dev libjpeg-dev libxml2-dev libxslt1-dev libffi-dev python-cairo libpango1.0-0

You need a recent *pip* for the installation of dependencies to work. If you
don't meet that requirement (Ubuntu trusty does not), run:

    pip install "pip>=1.5.6"

In any case, you then need to install coin python dependencies:

    pip install -r requirements.txt

You should now be able to run `python manage.py` (within the
virtualenv, obviously) without error.

Settings
--------

The `coin/settings_local.py` file is ignored by Git: feel free to override any
setting by writing into that file. For example, to override the `DEBUG`
settings:

    echo '# -*- coding: utf-8 -*-' > coin/settings_local.py
    echo 'DEBUG = TEMPLATE_DEBUG = True' >> coin/settings_local.py

See the end of this README for a reference of available configuration settings.

Database
--------

At this point, you should setup your database. You have two options.

### With PostgreSQL (for developpement), recomended

The official database for coin is postgresql.

To ease developpement, a postgresql virtual-machine recipe is provided
through [vagrant](https://vagrantup.com).


**Note: Vagrant is intended for developpement only and is totaly unsafe for a
production setup**.

Install requirements:

    sudo apt install virtualbox vagrant

Then, to boot and configure your dev VM:

    vagrant up

Default settings target that vagrant+postgreSQL setup, so, you don't have to
change any setting.


### With SQLite

SQLite setup may be simpler, but some features will not be available, namely:

- automatic allocation of IP subnets (needs proper subnet implementation in
  the database)
- sending automated emails to remind of expiring membership fee
  (needs aggregation on date fields, see Django doc)

To use sqlite instead of PostgreSQL, you have
to [override local settings](#settings) with someting like:

```python
DATABASES = {
    # Base de donnée du SI
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'coin.sqlite3',
        'USER': '', # Not needed for SQLite
        'PASSWORD': '', # Not needed for SQLite
        'HOST': '',  # Empty for localhost through domain sockets
        'PORT': '',  # Empty for default
    },
}
```

### For both PostgreSQL and SQLite

The first time, you need to create the database, create a superuser, and
import some base data to play with:

    python manage.py migrate
    python manage.py createsuperuser
    python manage.py loaddata offers ip_pool # skip this if you don't use PostgreSQL

Note that the superuser will be inserted into the LDAP backend exactly in the
same way as all other members, so you should use a real account (not just
admin/admin).

Then, at each code update, you will only need to update dependencies and apply
new migrations:

    pip install -r requirements.txt
    python manage.py migrate


At this point, Django should run correctly:

    python manage.py runserver


Running tests
-------------

There is a set of unit tests you can run with :

    DJANGO_SETTINGS_MODULE=coin.settings_test ./manage.py test

LDAP-related tests are disabled by default.

Setup LDAP parameters and activate LDAP in settings to make the LDAP tests
run.

### With pytest

Setup:

    pip install pytest-django

Run:

    pytest


Models graph
------------

*You need to have **graphviz** package installed on your system*.

You can build a huge PDF of the models used in coin with :

    ./manage.py graph_models \
      coin members offers billing resources reverse_dns configuration isp_database \
      -o models.pdf \
      --group-models

You can tweak the app list to graph (eg: if you use [optional
apps](#using-optional-apps)).

For example, if you want to graph only the `hardware_provisioning` app, run :

    ./manage.py graph_models \
      hardware_provisioning
      -o models.pdf \
      --group-models

And for more fine tunning of graph generation :

    ./manage.py graph_models --help

Available commands
==================

Some useful administration commands are available via `manage.py`.

`python manage.py members_email`: returns email addresses of all members, one
per line.  This may be useful to automatically feed a mailing list software.
Note that membership is based on the `status` field of users, not on
membership fees.  That is, even if a member has forgot to renew his or her
membership fee, his or her address will still show up in this list. More
filters are available, see the command's help for more details.

`python manage.py charge_subscriptions`: generate invoices (including a
PDF version) for each subscriber.  You probably want to run this command
every month as a cron task, see below.

`python manage.py call_for_membership_fees`: send reminder emails to members
whose membership fee is about to expire or is already expired (1 month before,
on the day of expiration, 1 month after, 2 months after, and 3 months after).
You should run this command in a cron job every day.

`python manage.py offer_subscriptions_count`: Returns subscription count grouped
by offer type.

`python manage.py import_payments_from_csv`: Import a CSV from a bank and match
payments with services and/or members. At the moment, this is quite specific to
ARN workflow

Configuration
=============

You should first setup the `sites` application, in the admin.  The domain name
configured there is used for outgoing emails.

LDAP
----

By default, LDAP support is disabled.  If you want to use LDAP (for
instance for the OpenVPN/LDAP backend, or for your own backend), see the
configuration file `coin/settings_local.example-illyse.py`.

ISP-specific configuration
--------------------------

The main place to customize Coin for your ISP is the IspDatabase application,
in the admin.  Information entered in this application has two purposes:

1) generate a JSON file usable by http://db.ffdn.org/

2) customize the user interface (postal address, bank account, ISP name & logo...)


Some bits of configuration are done in `settings.py`: LDAP branches, RSS feeds
to display on the home page, and so on.  Lastly, it is possible to override
all Coin templates (i.e. for user views and emails), see below.

Sending emails
--------------

Coin sends mails, you might want to customize the sender address:

- `DEFAULT_FROM_EMAIL` app-wide setting
- *administrative email* field, from *ISPDatabase* app, if set, will take
  precedence over `DEFAULT_FROM_EMAIL` for administrative emails (welcome email
  and membership fees), will take precedence (if filled).

Cron tasks
----------

You may want to run cron jobs for repetitive tasks.

The command that generates invoices should be run every day, so that invoices
for new subscriptions are created automatically.  On the first day of
each month, invoices will be generated for all subscriptions.
This will run the command at 23:59 every day:

    59 23 * * * /home/coin/venv/bin/python manage.py charge_subscriptions`

To send reminder emails for membership fee expiration:

    42 3 * * * /home/coin/venv/bin/python manage.py call_for_membership_fees`

If you want to be notified by email each time a reminder email is sent, you
can set an email address in the crontab:

    MAILTO=tresorier@myisp.fr

Customizing templates
---------------------

You may want to override some of the (HTML or email) templates to better suit
your structure needs.

Coin allows you to have a folder of custom templates that will contain your
templates, which gets loaded prior to coin builtins.

With this method, several templates can be overridden:

- template files in `coin/templates/`
- template files in `coin/<app_name>/templates/` for all active applications

### Do once (setup)

- Create a folder dedicated to your custom templates (*hint: outside of coin git
  tree is better*).
- Register that folder in the `EXTRA_TEMPLATE_DIRS` settings.

For instance, in `settings_local.py`:

    EXTRA_TEMPLATE_DIRS = ('/home/coin/my-folder/templates',)

### For each template you want to override

Copy the template you want to override to the right place in your custom
 folder (that's the hard part, see the example).

*Example*

Say we want to override the template located at
`coin/members/templates/members/emails/call_for_membership_fees.html` and we
set `EXTRA_TEMPLATE_DIRS = ('/home/coin/my-folder/templates',)` in settings.

Then  make a copy of the template file (and customize it) at
`/home/coin/my-folder/templates/members/emails/call_for_membership_fees.html`

Good to go :-)

Using optional apps
-------------------

Some apps are not enabled by default :

- *vpn*: Management of OpenVPN subscription and credentials through LDAP (used by [Illyse](https://www.illyse.net/) and [ARN](https://www.arn-fai.net/))
- *simple_dsl*: Simple DSL subscriptions, without handling
   any authentication backend or user configuration ("marque blanche")
- *dsl_ldap*: A DSL service which provisions LDAP for radius use (used by [Illyse](https://www.illyse.net/))
- *hardware_provisioning* : Self-service app to manage hardware inventory,
  hardware lent to members or in different stock sites.
- *maillists* : Self-service mailling-list (un)subscription for members:
  handles subscribers list for a third-party mailling-list server (sympa,
  mailman…
  etc). See [maillists app documentation (fr)](doc/admin/maillists.md).

You can enable them using the `EXTRA_INSTALLED_APPS` setting.
E.g. in `settings_local.py`:

    EXTRA_INSTALLED_APPS = (
        'vpn',
    )

If you enable an extra-app after initial installation, make sure to sync database :

    ./manage.py migrate

*nb: extra apps are loaded after the builtin apps.*


Settings
========

List of available settings in your `settings_local.py` file.

- `EXTRA_INSTALLED_APPS`: See [using optional apps](#using-optional-apps)
- `EXTRA_TEMPLATE_DIRS`: See [customizing templates](#customizing-templates)
- `LDAP_ACTIVATE`: See *LDAP*
- `MEMBER_MEMBERSHIP_INFO_URL`: Link to a page with information on how to become a member or pay the membership fee
- `MEMBERSHIP_FEE_REMINDER_DATES`: how long before/after the membership fee
  anniversary date we want to send a reminder email to the member. It defaults
  to the following:

```
MEMBERSHIP_FEE_REMINDER_DATES = [
    {'months': -3},  # 3 months before
    {'months': -2},  # 2 months before
    {'months': -1},  # 1 month before
    {'days': 0},     # the day of anniversary
    {'months': +1},  # 1 month after
]
```
- `SUBSCRIPTION_REFERENCE`: Pattern used to display a unique reference for any subscription. Helpful for bank wire transfer identification
- `REGISTRATION_OPEN` : Allow visitor to join the association by register on COIN
- `ACCOUNT_ACTIVATION_DAYS` : All account with unvalidated email will be deleted after X days
- `MEMBERSHIP_REFERENCE` : Template string to display the label the member should indicates for the bank transfer, default: "ADH-{{ user.pk }}"
- `DEFAULT_MEMBERSHIP_FEE` : Default membership fee, if you have a more complex membership fees policy, you could overwrite templates
- `PAYMENT_DELAY`: Payment delay in days for issued invoices ( default is 30 days which is the default in french law)
- `MEMBER_CAN_EDIT_PROFILE`: Allow members to edit their profiles
- `HANDLE_BALANCE`: Allows to handle money balances for members (False default)
- `INVOICES_INCLUDE_CONFIG_COMMENTS`: Add comment related to a subscription configuration when generating invoices
- `MEMBER_CAN_EDIT_VPN_CONF`: Allow members to edit some part of their vpn configuration
- `DEBUG` : Enable debug for development **do not use in production** : display
   stracktraces and enable [django-debug-toolbar](https://django-debug-toolbar.readthedocs.io).
- `SITE_TITLE`: the base of site title (displayed in browser window/tab title)
- `SITE_HEADER`: the site header (displayed in all pages as page header)
- `SITE_LOGO_URL`: URL to website logo (by default, this is an ascii-duck)
- `SUBSCRIPTIONS_NOTIFICATION_EMAILS` : Emails on which to send notifications
  uppon new registration.

### Optional apps settings

See also [using optional apps](#using-optional-apps).

#### maillists

- `MAILLIST_SYNC_COMMAND` : The command to send the list of mail addresses of a
  given mailling list to mail list server. The command will receives one
  address/line on stdin. This setting could use placholders:
    - `{email}`: the mail address of the list
    - `{short_name}`: the list name

#### vpn

- `VPN_SECRETS_TRANSMISSION_METHOD` : how are VPN secrets transmited to
  subscriber ? Two values are currently supported :
    - `gen-password-and-forget` (default, used by Illyse) : generate a
      password, push it to LDAP (which holds VPN auth), displays it to user and
      forget it.
    - `crypto-link` (used by ARN) : credentials are generated by an admin
      outside coin, and put on an encrypted burn-after-reading web page, whom
      URL is filled-in coin.


Accounting logs
---------------

To log 'accounting-related operations' (creation/update of invoice, payment
and member balance) to a specific file, add the following to settings_local.py :

```
from settings_base import *
LOGGING["formatters"]["verbose"] = {'format': "%(asctime)s - %(name)s - %(levelname)s - %(message)s"}
LOGGING["handlers"]["coin_accounting"] = {
    'level':'INFO',
    'class':'logging.handlers.RotatingFileHandler',
    'formatter': 'verbose',
    'filename': '/var/log/coin/accounting.log',
    'maxBytes': 1024*1024*15, # 15MB
    'backupCount': 10,
}
LOGGING["loggers"]["coin.billing"]["handlers"] = [ 'coin_accounting' ]
```

More information
================

For the rest of the setup (database, LDAP), see https://doc.illyse.net/projects/ils-si/wiki/Mise_en_place_environnement_de_dev

For real production deployment, see file `DEPLOYMENT.md`.