README.md 15.8 KB
Newer Older
1
2
3
The COIN project
================

alexAubin's avatar
alexAubin committed
4
COIN is an Information System designed for associative ISPs in the FFDN.
jocelyn's avatar
jocelyn committed
5
6
7

![example of a coin user view : the subscriptions list](doc/_img/user-subscriptions.png)

alexAubin's avatar
alexAubin committed
8
9
10
11
It is written in Django, and features :
- a generic configuration interface that correspond to services provided to members
- subscriptions management
- and the corresponding billing / payment handling system
Baptiste Jonglez's avatar
Baptiste Jonglez committed
12

alexAubin's avatar
alexAubin committed
13
14
15
COIN can can be interfaced with the actual infrastructure to fetch the state of
services (e.g. VPS, VPN, ...) and provision them via hooks defined in the
settings.
16

17
- The project page (issue, wiki, etc) is here: https://code.ffdn.org/FFDN/coin/
Baptiste Jonglez's avatar
Baptiste Jonglez committed
18

19
- A mailing list is available, for both users and developers: https://lists.ffdn.org/wws/info/coin
Baptiste Jonglez's avatar
Baptiste Jonglez committed
20

21
- IRC : #coin-dev sur freenode / Matrix: #freenode_#coin-dev:matrix.org
Baptiste Jonglez's avatar
Baptiste Jonglez committed
22

23
24
25
26
27
License
=======

Coin is released under the GNU Affero GPL 3.0 license.  See `LICENSE` for details.

28
29
30
31
32
33
Extending Coin
==============

If you want to write your own backend, see `EXTENDING.md`.


Baptiste Jonglez's avatar
Baptiste Jonglez committed
34
35
Quickstart
==========
36

37
38
39
40
41
Virtualenv
----------

Using a virtualenv is recommended.  On Debian, install `python-virtualenv`.
On Archlinux, the package is called `python2-virtualenv`, and you must
42
43
44
45
replace the `virtualenv` command with `virtualenv2` in the following.

To create the virtualenv (the first time):

jocelyn's avatar
jocelyn committed
46
    virtualenv ./venv
47
48
49
50
51


To activate the virtualenv (you need to do this each time you work on
the project):

jocelyn's avatar
jocelyn committed
52
    source ./venv/bin/activate
53
54
55


Install dependencies. On Debian, you will probably need the
56
57
`python-dev`, `python-pip`, `libldap-dev`, `libpq-dev`, `libsasl2-dev`,
and `libjpeg-dev` packages.
Fabs's avatar
Fabs committed
58

59
    sudo apt-get install python-dev python-pip libldap2-dev libpq-dev libsasl2-dev libjpeg-dev libxml2-dev libxslt1-dev libffi-dev python-cairo libpango1.0-0
Fabs's avatar
Fabs committed
60

jocelyn's avatar
jocelyn committed
61
62
You need a recent *pip* for the installation of dependencies to work. If you
don't meet that requirement (Ubuntu trusty does not), run:
63

jocelyn's avatar
jocelyn committed
64
    pip install "pip>=1.5.6"
65
66

In any case, you then need to install coin python dependencies:
67

68
    pip install -r requirements.txt
69
70
71

You should now be able to run `python manage.py` (within the
virtualenv, obviously) without error.
Baptiste Jonglez's avatar
Baptiste Jonglez committed
72

73
74
75
Settings
--------

76
The `coin/settings_local.py` file is ignored by Git: feel free to override any
77
78
79
80
81
82
setting by writing into that file. For example, to override the `DEBUG`
settings:

    echo '# -*- coding: utf-8 -*-' > coin/settings_local.py
    echo 'DEBUG = TEMPLATE_DEBUG = True' >> coin/settings_local.py

83
84
See the end of this README for a reference of available configuration settings.

85
86
Development
-----------
87

88
A Dockerfile + docker-compose.yml is provided **for development purposes only**.
89

90
- Install Docker according to Docker's documentation : https://docs.docker.com/engine/install/ (or by running `apt install docker`)
91
92
- Validate that docker is running with `sudo systemctl status docker`
- You may need to add your user to the docker group to be able to run docker commands as non-root:
jocelyn's avatar
jocelyn committed
93

94
95
96
97
```
sudo usermod -aG docker $USER
# Then you need to reload your entire session (not just the terminal) for this to propagate...
# or maybe running this command does the trick: newgrp docker
jocelyn's avatar
jocelyn committed
98
99
```

100
- Install docker-compose: `sudo pip3 install docker-compose`
alexAubin's avatar
alexAubin committed
101
- Run `docker-compose up` ... the first time this will download and build a whole bunch of stuff
102
- You can access your app through http://localhost:8000/
103
- Misc tips:
104

105
```
alexAubin's avatar
alexAubin committed
106
107
# Launch docker-compose up, but in background
docker-compose up -d
Baptiste Jonglez's avatar
Baptiste Jonglez committed
108

109
110
# Check container logs after startup
docker-compose logs -f coin
Fabs's avatar
Fabs committed
111

112
113
# Stop / relaunch the container
docker-compose stop
alexAubin's avatar
alexAubin committed
114
docker-compose up
Fabs's avatar
Fabs committed
115

116
# Regen and apply migrations
alexAubin's avatar
alexAubin committed
117
118
# (ugly hack: the user coin can't write in the mounted folder ... so gotta run this as root..)
docker-compose exec -u root coin python manage.py makemigrations
119
docker-compose exec coin python manage.py migrate
120

121
122
123
# Enter a shell inside the container
docker-compose exec coin bash
```
124

125
- At some point you will want to initialize the admin and a bunch of dummy data :
Fabs's avatar
Fabs committed
126

127
128
129
130
```
docker-compose exec coin python manage.py fill_with_toy_data
docker-compose exec coin python manage.py changepassword admin
```
Fabs's avatar
Fabs committed
131

jocelyn's avatar
jocelyn committed
132
133
134
135
136
Running tests
-------------

There is a set of unit tests you can run with :

137
    DJANGO_SETTINGS_MODULE=coin.settings_test ./manage.py test
jocelyn's avatar
jocelyn committed
138
139
140
141
142
143

LDAP-related tests are disabled by default.

Setup LDAP parameters and activate LDAP in settings to make the LDAP tests
run.

jocelyn's avatar
jocelyn committed
144
145
146
147
148
149
150
151
152
153
### With pytest

Setup:

    pip install pytest-django

Run:

    pytest

jocelyn's avatar
jocelyn committed
154

jocelyn's avatar
jocelyn committed
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
Models graph
------------

*You need to have **graphviz** package installed on your system*.

You can build a huge PDF of the models used in coin with :

    ./manage.py graph_models \
      coin members offers billing resources reverse_dns configuration isp_database \
      -o models.pdf \
      --group-models

You can tweak the app list to graph (eg: if you use [optional
apps](#using-optional-apps)).

For example, if you want to graph only the `hardware_provisioning` app, run :

    ./manage.py graph_models \
      hardware_provisioning
      -o models.pdf \
      --group-models

And for more fine tunning of graph generation :

    ./manage.py graph_models --help

181
182
183
184
185
186
187
188
189
Available commands
==================

Some useful administration commands are available via `manage.py`.

`python manage.py members_email`: returns email addresses of all members, one
per line.  This may be useful to automatically feed a mailing list software.
Note that membership is based on the `status` field of users, not on
membership fees.  That is, even if a member has forgot to renew his or her
190
191
membership fee, his or her address will still show up in this list. More
filters are available, see the command's help for more details.
192
193
194
195
196

`python manage.py charge_subscriptions`: generate invoices (including a
PDF version) for each subscriber.  You probably want to run this command
every month as a cron task, see below.

197
198
199
200
201
`python manage.py call_for_membership_fees`: send reminder emails to members
whose membership fee is about to expire or is already expired (1 month before,
on the day of expiration, 1 month after, 2 months after, and 3 months after).
You should run this command in a cron job every day.

opi's avatar
opi committed
202
203
`python manage.py offer_subscriptions_count`: Returns subscription count grouped
by offer type.
204

205
206
207
208
`python manage.py import_payments_from_csv`: Import a CSV from a bank and match
payments with services and/or members. At the moment, this is quite specific to
ARN workflow

209
210
211
212
213
214
Configuration
=============

You should first setup the `sites` application, in the admin.  The domain name
configured there is used for outgoing emails.

215
216
217
218
219
220
221
LDAP
----

By default, LDAP support is disabled.  If you want to use LDAP (for
instance for the OpenVPN/LDAP backend, or for your own backend), see the
configuration file `coin/settings_local.example-illyse.py`.

222
ISP-specific configuration
223
--------------------------
224
225
226
227
228
229
230
231
232
233

The main place to customize Coin for your ISP is the IspDatabase application,
in the admin.  Information entered in this application has two purposes:

1) generate a JSON file usable by http://db.ffdn.org/

2) customize the user interface (postal address, bank account, ISP name & logo...)


Some bits of configuration are done in `settings.py`: LDAP branches, RSS feeds
234
235
to display on the home page, and so on.  Lastly, it is possible to override
all Coin templates (i.e. for user views and emails), see below.
236

237
238
239
240
241
242
243
244
245
246
Sending emails
--------------

Coin sends mails, you might want to customize the sender address:

- `DEFAULT_FROM_EMAIL` app-wide setting
- *administrative email* field, from *ISPDatabase* app, if set, will take
  precedence over `DEFAULT_FROM_EMAIL` for administrative emails (welcome email
  and membership fees), will take precedence (if filled).

247
248
249
250
251
Cron tasks
----------

You may want to run cron jobs for repetitive tasks.

252
253
254
255
The command that generates invoices should be run every day, so that invoices
for new subscriptions are created automatically.  On the first day of
each month, invoices will be generated for all subscriptions.
This will run the command at 23:59 every day:
256

257
    59 23 * * * /home/coin/venv/bin/python manage.py charge_subscriptions`
258

259
260
To send reminder emails for membership fee expiration:

261
    42 3 * * * /home/coin/venv/bin/python manage.py call_for_membership_fees`
262

263
264
265
266
If you want to be notified by email each time a reminder email is sent, you
can set an email address in the crontab:

    MAILTO=tresorier@myisp.fr
267

jocelyn's avatar
jocelyn committed
268
269
270
271
272
273
274
275
276
Customizing templates
---------------------

You may want to override some of the (HTML or email) templates to better suit
your structure needs.

Coin allows you to have a folder of custom templates that will contain your
templates, which gets loaded prior to coin builtins.

277
With this method, several templates can be overridden:
jocelyn's avatar
jocelyn committed
278

279
280
- template files in `coin/templates/`
- template files in `coin/<app_name>/templates/` for all active applications
jocelyn's avatar
jocelyn committed
281
282
283
284
285

### Do once (setup)

- Create a folder dedicated to your custom templates (*hint: outside of coin git
  tree is better*).
286
287
288
- Register that folder in the `EXTRA_TEMPLATE_DIRS` settings.

For instance, in `settings_local.py`:
jocelyn's avatar
jocelyn committed
289
290
291
292
293
294
295
296
297
298

    EXTRA_TEMPLATE_DIRS = ('/home/coin/my-folder/templates',)

### For each template you want to override

Copy the template you want to override to the right place in your custom
 folder (that's the hard part, see the example).

*Example*

299
Say we want to override the template located at
jocelyn's avatar
jocelyn committed
300
301
302
303
304
305
306
307
`coin/members/templates/members/emails/call_for_membership_fees.html` and we
set `EXTRA_TEMPLATE_DIRS = ('/home/coin/my-folder/templates',)` in settings.

Then  make a copy of the template file (and customize it) at
`/home/coin/my-folder/templates/members/emails/call_for_membership_fees.html`

Good to go :-)

jocelyn's avatar
jocelyn committed
308
309
310
311
312
Using optional apps
-------------------

Some apps are not enabled by default :

ljf's avatar
ljf committed
313
- *vpn*: Management of OpenVPN subscription and credentials through LDAP (used by [Illyse](https://www.illyse.net/) and [ARN](https://www.arn-fai.net/))
314
315
- *simple_dsl*: Simple DSL subscriptions, without handling
   any authentication backend or user configuration ("marque blanche")
jocelyn's avatar
jocelyn committed
316
- *dsl_ldap*: A DSL service which provisions LDAP for radius use (used by [Illyse](https://www.illyse.net/))
jocelyn's avatar
jocelyn committed
317
318
- *hardware_provisioning* : Self-service app to manage hardware inventory,
  hardware lent to members or in different stock sites.
jocelyn's avatar
jocelyn committed
319
320
321
322
- *maillists* : Self-service mailling-list (un)subscription for members:
  handles subscribers list for a third-party mailling-list server (sympa,
  mailman…
  etc). See [maillists app documentation (fr)](doc/admin/maillists.md).
jocelyn's avatar
jocelyn committed
323

Baptiste Jonglez's avatar
Baptiste Jonglez committed
324
325
You can enable them using the `EXTRA_INSTALLED_APPS` setting.
E.g. in `settings_local.py`:
jocelyn's avatar
jocelyn committed
326
327

    EXTRA_INSTALLED_APPS = (
Baptiste Jonglez's avatar
Baptiste Jonglez committed
328
329
        'vpn',
    )
jocelyn's avatar
jocelyn committed
330
331
332
333
334
335
336
337

If you enable an extra-app after initial installation, make sure to sync database :

    ./manage.py migrate

*nb: extra apps are loaded after the builtin apps.*


338
339
340
341
342
Settings
========

List of available settings in your `settings_local.py` file.

jocelyn's avatar
jocelyn committed
343
344
- `EXTRA_INSTALLED_APPS`: See [using optional apps](#using-optional-apps)
- `EXTRA_TEMPLATE_DIRS`: See [customizing templates](#customizing-templates)
345
- `LDAP_ACTIVATE`: See *LDAP*
346
- `MEMBER_MEMBERSHIP_INFO_URL`: Link to a page with information on how to become a member or pay the membership fee
347
348
349
350
351
352
353
354
355
356
357
358
359
- `MEMBERSHIP_FEE_REMINDER_DATES`: how long before/after the membership fee
  anniversary date we want to send a reminder email to the member. It defaults
  to the following:

```
MEMBERSHIP_FEE_REMINDER_DATES = [
    {'months': -3},  # 3 months before
    {'months': -2},  # 2 months before
    {'months': -1},  # 1 month before
    {'days': 0},     # the day of anniversary
    {'months': +1},  # 1 month after
]
```
opi's avatar
opi committed
360
- `SUBSCRIPTION_REFERENCE`: Pattern used to display a unique reference for any subscription. Helpful for bank wire transfer identification
ljf's avatar
ljf committed
361
362
363
- `REGISTRATION_OPEN` : Allow visitor to join the association by register on COIN
- `ACCOUNT_ACTIVATION_DAYS` : All account with unvalidated email will be deleted after X days
- `MEMBERSHIP_REFERENCE` : Template string to display the label the member should indicates for the bank transfer, default: "ADH-{{ user.pk }}"
ljf's avatar
ljf committed
364
- `DEFAULT_MEMBERSHIP_FEE` : Default membership fee, if you have a more complex membership fees policy, you could overwrite templates
365
- `PAYMENT_DELAY`: Payment delay in days for issued invoices ( default is 30 days which is the default in french law)
366
- `MEMBER_TERMS` : You can ask for new user to accept a terms during registration. (example:  'J\'ai lu les <a href="https://wiki.arn-fai.net/administratif:statuts" target="_BLANK">statuts de l\'association</a>' )
367
- `MEMBER_CAN_EDIT_PROFILE`: Allow members to edit their profiles
368
- `HANDLE_BALANCE`: Allows to handle money balances for members (False default)
369
- `INVOICES_INCLUDE_CONFIG_COMMENTS`: Add comment related to a subscription configuration when generating invoices
ljf's avatar
ljf committed
370
- `MEMBER_CAN_EDIT_VPN_CONF`: Allow members to edit some part of their vpn configuration
371
- `IP_ALLOCATION_MESSAGE`: Template string that will be used to log IP allocation in the corresponding coin.subnets logging system, defaults to `None` which is disabled logging, full example : `"{ip} to {member.pk} ({member.username} - {member.first_name} {member.last_name}) (for offer {offer}, {ref})"`
372
373
- `DEBUG` : Enable debug for development **do not use in production** : display
   stracktraces and enable [django-debug-toolbar](https://django-debug-toolbar.readthedocs.io).
374
375
- `SITE_TITLE`: the base of site title (displayed in browser window/tab title)
- `SITE_HEADER`: the site header (displayed in all pages as page header)
jocelyn's avatar
jocelyn committed
376
- `SITE_LOGO_URL`: URL to website logo (by default, this is an ascii-duck)
jocelyn's avatar
jocelyn committed
377
378
- `SUBSCRIPTIONS_NOTIFICATION_EMAILS` : Emails on which to send notifications
  uppon new registration.
379

380
381
382
383
384
385
386
387
388
389
390
391
### Optional apps settings

See also [using optional apps](#using-optional-apps).

#### maillists

- `MAILLIST_SYNC_COMMAND` : The command to send the list of mail addresses of a
  given mailling list to mail list server. The command will receives one
  address/line on stdin. This setting could use placholders:
    - `{email}`: the mail address of the list
    - `{short_name}`: the list name

392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
Accounting logs
---------------

To log 'accounting-related operations' (creation/update of invoice, payment
and member balance) to a specific file, add the following to settings_local.py :

```
from settings_base import *
LOGGING["formatters"]["verbose"] = {'format': "%(asctime)s - %(name)s - %(levelname)s - %(message)s"}
LOGGING["handlers"]["coin_accounting"] = {
    'level':'INFO',
    'class':'logging.handlers.RotatingFileHandler',
    'formatter': 'verbose',
    'filename': '/var/log/coin/accounting.log',
    'maxBytes': 1024*1024*15, # 15MB
    'backupCount': 10,
}
LOGGING["loggers"]["coin.billing"]["handlers"] = [ 'coin_accounting' ]
```
411

ljf's avatar
ljf committed
412
413
External account
---------------
Baptiste Jonglez's avatar
Baptiste Jonglez committed
414

ljf's avatar
ljf committed
415
416
 - `VERBOSE_NAME_EXTERNAL_ACCOUNT` : This label is display in the member subscription view (ARN put this to 'Compte sans-nuage')
 - `VERBOSE_NAME_PLURAL_EXTERNAL_ACCOUNT` : same for plural
Baptiste Jonglez's avatar
Baptiste Jonglez committed
417
418


419
420
Deployment
==========
421

Baptiste Jonglez's avatar
Baptiste Jonglez committed
422
For real production deployment, see file `DEPLOYMENT.md`.
423
424
425
426
427
428
429
430
431
432

Testing an upgrade in a test env prior to upgrading your production
===================================================================

When upgrading from a super-old version of Coin, you may want to test the upgrade in a test env instead of yoloupgrading the production ;)

Here are some guidelines:

- Obtain a dump from your Coin's production DB
- Clone this repository, install Docker (c.f. 'Development' section)
433
- Make sure that you tweak the settings file to enable your relevant modules
434
435
436
437
- `mkdir ./dumps/` in the git clone, add your `.sql` dump inside, then mount it in the db's container by adding a line such as `- ./dumps/:/mnt/dumps/` in `db` > `volumes` inside `docker-compose.yml`
- Launch the `db` container with `docker-compose up -d db`, enter the container with `docker-compose exec -u root db bash`, then restore the dump with `psql -U coin coin < /mnt/dumps/yourdump.sql`
- Let's now enter the app container with `docker-compose exec coin bash`, then run migrations with `./manage.py migrate`
- Check that everything is okay (going to `http://localhost:8000/`), and/or debug if needed ;)