README.md 15.1 KB
Newer Older
1 2 3
The COIN project
================

Baptiste Jonglez's avatar
Baptiste Jonglez committed
4 5
`Coin` is Illyse's Information System, designed to manage both members
and Internet accesses, such as through DSL, VPN, wireless…
6

jocelyn's avatar
jocelyn committed
7 8 9

![example of a coin user view : the subscriptions list](doc/_img/user-subscriptions.png)

10 11 12 13 14
It is written in Django, and features a generic configuration interface,
which allows to implement custom backends for different technologies.
Currently implemented is a LDAP-based backend for OpenVPN, and a very
simple DSL backend, without any authentication (useful for "white label"
DSL).
Baptiste Jonglez's avatar
Baptiste Jonglez committed
15 16

Coin currently only works with python2, because `python-ldap` is (as of
17 18
2013) not compatible with python3.

Baptiste Jonglez's avatar
Baptiste Jonglez committed
19 20
The project page (issue, wiki, etc) is here:

21
  https://code.ffdn.org/FFDN/coin/
Baptiste Jonglez's avatar
Baptiste Jonglez committed
22

Baptiste Jonglez's avatar
Baptiste Jonglez committed
23 24
A mailing list is available, for both users and developers: https://listes.illyse.org/wws/info/si

25 26 27 28 29
License
=======

Coin is released under the GNU Affero GPL 3.0 license.  See `LICENSE` for details.

30 31 32 33 34 35
Extending Coin
==============

If you want to write your own backend, see `EXTENDING.md`.


Baptiste Jonglez's avatar
Baptiste Jonglez committed
36 37
Quickstart
==========
38

39 40 41 42 43
Virtualenv
----------

Using a virtualenv is recommended.  On Debian, install `python-virtualenv`.
On Archlinux, the package is called `python2-virtualenv`, and you must
44 45 46 47
replace the `virtualenv` command with `virtualenv2` in the following.

To create the virtualenv (the first time):

jocelyn's avatar
jocelyn committed
48
    virtualenv ./venv
49 50 51 52 53


To activate the virtualenv (you need to do this each time you work on
the project):

jocelyn's avatar
jocelyn committed
54
    source ./venv/bin/activate
55 56 57


Install dependencies. On Debian, you will probably need the
58 59
`python-dev`, `python-pip`, `libldap-dev`, `libpq-dev`, `libsasl2-dev`,
and `libjpeg-dev` packages.
Fabs's avatar
Fabs committed
60

61
    sudo apt-get install python-dev python-pip libldap2-dev libpq-dev libsasl2-dev libjpeg-dev libxml2-dev libxslt1-dev libffi-dev python-cairo libpango1.0-0
Fabs's avatar
Fabs committed
62

jocelyn's avatar
jocelyn committed
63 64
You need a recent *pip* for the installation of dependencies to work. If you
don't meet that requirement (Ubuntu trusty does not), run:
65

jocelyn's avatar
jocelyn committed
66
    pip install "pip>=1.5.6"
67 68

In any case, you then need to install coin python dependencies:
69

70
    pip install -r requirements.txt
71 72 73

You should now be able to run `python manage.py` (within the
virtualenv, obviously) without error.
Baptiste Jonglez's avatar
Baptiste Jonglez committed
74

75 76 77
Settings
--------

78
The `coin/settings_local.py` file is ignored by Git: feel free to override any
79 80 81 82 83 84
setting by writing into that file. For example, to override the `DEBUG`
settings:

    echo '# -*- coding: utf-8 -*-' > coin/settings_local.py
    echo 'DEBUG = TEMPLATE_DEBUG = True' >> coin/settings_local.py

85 86 87 88 89
See the end of this README for a reference of available configuration settings.

Database
--------

jocelyn's avatar
jocelyn committed
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117
At this point, you should setup your database. You have two options.

### With PostgreSQL (for developpement), recomended

The official database for coin is postgresql.

To ease developpement, a postgresql virtual-machine recipe is provided
through [vagrant](https://vagrantup.com).


**Note: Vagrant is intended for developpement only and is totaly unsafe for a
production setup**.

Install requirements:

    sudo apt install virtualbox vagrant

Then, to boot and configure your dev VM:

    vagrant up

Default settings target that vagrant+postgreSQL setup, so, you don't have to
change any setting.


### With SQLite

SQLite setup may be simpler, but some features will not be available, namely:
118

119 120 121 122
- automatic allocation of IP subnets (needs proper subnet implementation in
  the database)
- sending automated emails to remind of expiring membership fee
  (needs aggregation on date fields, see Django doc)
123

jocelyn's avatar
jocelyn committed
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141
To use sqlite instead of PostgreSQL, you have
to [override local settings](#settings) with someting like:

```python
DATABASES = {
    # Base de donnée du SI
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'coin.sqlite3',
        'USER': '', # Not needed for SQLite
        'PASSWORD': '', # Not needed for SQLite
        'HOST': '',  # Empty for localhost through domain sockets
        'PORT': '',  # Empty for default
    },
}
```

### For both PostgreSQL and SQLite
142

143 144
The first time, you need to create the database, create a superuser, and
import some base data to play with:
145

146 147
    python manage.py migrate
    python manage.py createsuperuser
148
    python manage.py loaddata offers ip_pool # skip this if you don't use PostgreSQL
149

Baptiste Jonglez's avatar
Baptiste Jonglez committed
150 151 152 153
Note that the superuser will be inserted into the LDAP backend exactly in the
same way as all other members, so you should use a real account (not just
admin/admin).

154 155
Then, at each code update, you will only need to update dependencies and apply
new migrations:
Fabs's avatar
Fabs committed
156

157
    pip install -r requirements.txt
158
    python manage.py migrate
Fabs's avatar
Fabs committed
159

160 161 162 163

At this point, Django should run correctly:

    python manage.py runserver
Fabs's avatar
Fabs committed
164 165


jocelyn's avatar
jocelyn committed
166 167 168 169 170
Running tests
-------------

There is a set of unit tests you can run with :

171
    DJANGO_SETTINGS_MODULE=coin.settings_test ./manage.py test
jocelyn's avatar
jocelyn committed
172 173 174 175 176 177

LDAP-related tests are disabled by default.

Setup LDAP parameters and activate LDAP in settings to make the LDAP tests
run.

jocelyn's avatar
jocelyn committed
178 179 180 181 182 183 184 185 186 187
### With pytest

Setup:

    pip install pytest-django

Run:

    pytest

jocelyn's avatar
jocelyn committed
188

jocelyn's avatar
jocelyn committed
189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214
Models graph
------------

*You need to have **graphviz** package installed on your system*.

You can build a huge PDF of the models used in coin with :

    ./manage.py graph_models \
      coin members offers billing resources reverse_dns configuration isp_database \
      -o models.pdf \
      --group-models

You can tweak the app list to graph (eg: if you use [optional
apps](#using-optional-apps)).

For example, if you want to graph only the `hardware_provisioning` app, run :

    ./manage.py graph_models \
      hardware_provisioning
      -o models.pdf \
      --group-models

And for more fine tunning of graph generation :

    ./manage.py graph_models --help

215 216 217 218 219 220 221 222 223
Available commands
==================

Some useful administration commands are available via `manage.py`.

`python manage.py members_email`: returns email addresses of all members, one
per line.  This may be useful to automatically feed a mailing list software.
Note that membership is based on the `status` field of users, not on
membership fees.  That is, even if a member has forgot to renew his or her
224 225
membership fee, his or her address will still show up in this list. More
filters are available, see the command's help for more details.
226 227 228 229 230

`python manage.py charge_subscriptions`: generate invoices (including a
PDF version) for each subscriber.  You probably want to run this command
every month as a cron task, see below.

231 232 233 234 235
`python manage.py call_for_membership_fees`: send reminder emails to members
whose membership fee is about to expire or is already expired (1 month before,
on the day of expiration, 1 month after, 2 months after, and 3 months after).
You should run this command in a cron job every day.

opi's avatar
opi committed
236 237
`python manage.py offer_subscriptions_count`: Returns subscription count grouped
by offer type.
238

239 240 241 242
`python manage.py import_payments_from_csv`: Import a CSV from a bank and match
payments with services and/or members. At the moment, this is quite specific to
ARN workflow

243 244 245 246 247 248
Configuration
=============

You should first setup the `sites` application, in the admin.  The domain name
configured there is used for outgoing emails.

249 250 251 252 253 254 255
LDAP
----

By default, LDAP support is disabled.  If you want to use LDAP (for
instance for the OpenVPN/LDAP backend, or for your own backend), see the
configuration file `coin/settings_local.example-illyse.py`.

256
ISP-specific configuration
257
--------------------------
258 259 260 261 262 263 264 265 266 267

The main place to customize Coin for your ISP is the IspDatabase application,
in the admin.  Information entered in this application has two purposes:

1) generate a JSON file usable by http://db.ffdn.org/

2) customize the user interface (postal address, bank account, ISP name & logo...)


Some bits of configuration are done in `settings.py`: LDAP branches, RSS feeds
268 269
to display on the home page, and so on.  Lastly, it is possible to override
all Coin templates (i.e. for user views and emails), see below.
270

271 272 273 274 275 276 277 278 279 280
Sending emails
--------------

Coin sends mails, you might want to customize the sender address:

- `DEFAULT_FROM_EMAIL` app-wide setting
- *administrative email* field, from *ISPDatabase* app, if set, will take
  precedence over `DEFAULT_FROM_EMAIL` for administrative emails (welcome email
  and membership fees), will take precedence (if filled).

281 282 283 284 285
Cron tasks
----------

You may want to run cron jobs for repetitive tasks.

286 287 288 289
The command that generates invoices should be run every day, so that invoices
for new subscriptions are created automatically.  On the first day of
each month, invoices will be generated for all subscriptions.
This will run the command at 23:59 every day:
290

291
    59 23 * * * /home/coin/venv/bin/python manage.py charge_subscriptions`
292

293 294
To send reminder emails for membership fee expiration:

295
    42 3 * * * /home/coin/venv/bin/python manage.py call_for_membership_fees`
296

297 298 299 300
If you want to be notified by email each time a reminder email is sent, you
can set an email address in the crontab:

    MAILTO=tresorier@myisp.fr
301

jocelyn's avatar
jocelyn committed
302 303 304 305 306 307 308 309 310
Customizing templates
---------------------

You may want to override some of the (HTML or email) templates to better suit
your structure needs.

Coin allows you to have a folder of custom templates that will contain your
templates, which gets loaded prior to coin builtins.

311
With this method, several templates can be overridden:
jocelyn's avatar
jocelyn committed
312

313 314
- template files in `coin/templates/`
- template files in `coin/<app_name>/templates/` for all active applications
jocelyn's avatar
jocelyn committed
315 316 317 318 319

### Do once (setup)

- Create a folder dedicated to your custom templates (*hint: outside of coin git
  tree is better*).
320 321 322
- Register that folder in the `EXTRA_TEMPLATE_DIRS` settings.

For instance, in `settings_local.py`:
jocelyn's avatar
jocelyn committed
323 324 325 326 327 328 329 330 331 332

    EXTRA_TEMPLATE_DIRS = ('/home/coin/my-folder/templates',)

### For each template you want to override

Copy the template you want to override to the right place in your custom
 folder (that's the hard part, see the example).

*Example*

333
Say we want to override the template located at
jocelyn's avatar
jocelyn committed
334 335 336 337 338 339 340 341
`coin/members/templates/members/emails/call_for_membership_fees.html` and we
set `EXTRA_TEMPLATE_DIRS = ('/home/coin/my-folder/templates',)` in settings.

Then  make a copy of the template file (and customize it) at
`/home/coin/my-folder/templates/members/emails/call_for_membership_fees.html`

Good to go :-)

jocelyn's avatar
jocelyn committed
342 343 344 345 346
Using optional apps
-------------------

Some apps are not enabled by default :

ljf's avatar
ljf committed
347
- *vpn*: Management of OpenVPN subscription and credentials through LDAP (used by [Illyse](https://www.illyse.net/) and [ARN](https://www.arn-fai.net/))
348 349
- *simple_dsl*: Simple DSL subscriptions, without handling
   any authentication backend or user configuration ("marque blanche")
jocelyn's avatar
jocelyn committed
350
- *dsl_ldap*: A DSL service which provisions LDAP for radius use (used by [Illyse](https://www.illyse.net/))
jocelyn's avatar
jocelyn committed
351 352
- *hardware_provisioning* : Self-service app to manage hardware inventory,
  hardware lent to members or in different stock sites.
jocelyn's avatar
jocelyn committed
353 354 355 356
- *maillists* : Self-service mailling-list (un)subscription for members:
  handles subscribers list for a third-party mailling-list server (sympa,
  mailman…
  etc). See [maillists app documentation (fr)](doc/admin/maillists.md).
jocelyn's avatar
jocelyn committed
357

Baptiste Jonglez's avatar
Baptiste Jonglez committed
358 359
You can enable them using the `EXTRA_INSTALLED_APPS` setting.
E.g. in `settings_local.py`:
jocelyn's avatar
jocelyn committed
360 361

    EXTRA_INSTALLED_APPS = (
Baptiste Jonglez's avatar
Baptiste Jonglez committed
362 363
        'vpn',
    )
jocelyn's avatar
jocelyn committed
364 365 366 367 368 369 370 371

If you enable an extra-app after initial installation, make sure to sync database :

    ./manage.py migrate

*nb: extra apps are loaded after the builtin apps.*


372 373 374 375 376
Settings
========

List of available settings in your `settings_local.py` file.

jocelyn's avatar
jocelyn committed
377 378
- `EXTRA_INSTALLED_APPS`: See [using optional apps](#using-optional-apps)
- `EXTRA_TEMPLATE_DIRS`: See [customizing templates](#customizing-templates)
379
- `LDAP_ACTIVATE`: See *LDAP*
380
- `MEMBER_MEMBERSHIP_INFO_URL`: Link to a page with information on how to become a member or pay the membership fee
381 382 383 384 385 386 387 388 389 390 391 392 393
- `MEMBERSHIP_FEE_REMINDER_DATES`: how long before/after the membership fee
  anniversary date we want to send a reminder email to the member. It defaults
  to the following:

```
MEMBERSHIP_FEE_REMINDER_DATES = [
    {'months': -3},  # 3 months before
    {'months': -2},  # 2 months before
    {'months': -1},  # 1 month before
    {'days': 0},     # the day of anniversary
    {'months': +1},  # 1 month after
]
```
opi's avatar
opi committed
394
- `SUBSCRIPTION_REFERENCE`: Pattern used to display a unique reference for any subscription. Helpful for bank wire transfer identification
ljf's avatar
ljf committed
395 396 397
- `REGISTRATION_OPEN` : Allow visitor to join the association by register on COIN
- `ACCOUNT_ACTIVATION_DAYS` : All account with unvalidated email will be deleted after X days
- `MEMBERSHIP_REFERENCE` : Template string to display the label the member should indicates for the bank transfer, default: "ADH-{{ user.pk }}"
ljf's avatar
ljf committed
398
- `DEFAULT_MEMBERSHIP_FEE` : Default membership fee, if you have a more complex membership fees policy, you could overwrite templates
399
- `PAYMENT_DELAY`: Payment delay in days for issued invoices ( default is 30 days which is the default in french law)
400
- `MEMBER_CAN_EDIT_PROFILE`: Allow members to edit their profiles
401
- `HANDLE_BALANCE`: Allows to handle money balances for members (False default)
402
- `INVOICES_INCLUDE_CONFIG_COMMENTS`: Add comment related to a subscription configuration when generating invoices
ljf's avatar
ljf committed
403
- `MEMBER_CAN_EDIT_VPN_CONF`: Allow members to edit some part of their vpn configuration
404 405
- `DEBUG` : Enable debug for development **do not use in production** : display
   stracktraces and enable [django-debug-toolbar](https://django-debug-toolbar.readthedocs.io).
406 407
- `SITE_TITLE`: the base of site title (displayed in browser window/tab title)
- `SITE_HEADER`: the site header (displayed in all pages as page header)
jocelyn's avatar
jocelyn committed
408
- `SITE_LOGO_URL`: URL to website logo (by default, this is an ascii-duck)
jocelyn's avatar
jocelyn committed
409 410
- `SUBSCRIPTIONS_NOTIFICATION_EMAILS` : Emails on which to send notifications
  uppon new registration.
411

412 413 414 415 416 417 418 419 420 421 422 423
### Optional apps settings

See also [using optional apps](#using-optional-apps).

#### maillists

- `MAILLIST_SYNC_COMMAND` : The command to send the list of mail addresses of a
  given mailling list to mail list server. The command will receives one
  address/line on stdin. This setting could use placholders:
    - `{email}`: the mail address of the list
    - `{short_name}`: the list name

424 425 426 427 428 429 430 431 432 433 434 435
#### vpn

- `VPN_SECRETS_TRANSMISSION_METHOD` : how are VPN secrets transmited to
  subscriber ? Two values are currently supported :
    - `gen-password-and-forget` (default, used by Illyse) : generate a
      password, push it to LDAP (which holds VPN auth), displays it to user and
      forget it.
    - `crypto-link` (used by ARN) : credentials are generated by an admin
      outside coin, and put on an encrypted burn-after-reading web page, whom
      URL is filled-in coin.


436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454
Accounting logs
---------------

To log 'accounting-related operations' (creation/update of invoice, payment
and member balance) to a specific file, add the following to settings_local.py :

```
from settings_base import *
LOGGING["formatters"]["verbose"] = {'format': "%(asctime)s - %(name)s - %(levelname)s - %(message)s"}
LOGGING["handlers"]["coin_accounting"] = {
    'level':'INFO',
    'class':'logging.handlers.RotatingFileHandler',
    'formatter': 'verbose',
    'filename': '/var/log/coin/accounting.log',
    'maxBytes': 1024*1024*15, # 15MB
    'backupCount': 10,
}
LOGGING["loggers"]["coin.billing"]["handlers"] = [ 'coin_accounting' ]
```
455

Baptiste Jonglez's avatar
Baptiste Jonglez committed
456 457 458
More information
================

jocelyn's avatar
jocelyn committed
459
For the rest of the setup (database, LDAP), see https://doc.illyse.net/projects/ils-si/wiki/Mise_en_place_environnement_de_dev
Baptiste Jonglez's avatar
Baptiste Jonglez committed
460 461

For real production deployment, see file `DEPLOYMENT.md`.