diff options
| -rw-r--r-- | docs/source/siteadmin/deploying.rst | 68 | ||||
| -rw-r--r-- | docs/source/siteadmin/production-deployments.rst | 147 |
2 files changed, 79 insertions, 136 deletions
diff --git a/docs/source/siteadmin/deploying.rst b/docs/source/siteadmin/deploying.rst index de74479a..e7dc78f0 100644 --- a/docs/source/siteadmin/deploying.rst +++ b/docs/source/siteadmin/deploying.rst @@ -498,10 +498,10 @@ Type ``Ctrl-c`` to exit the above server test and ``exit`` or Run MediaGoblin as a system service ----------------------------------- -To ensure MediaGoblin is automatically started and restarted in case of -problems, we need to run it as a system service. If your operating system uses -Systemd, you can use Systemd ``service files`` to manage both the Celery and -Paste processes. +To ensure MediaGoblin is automatically started and restarted in case +of problems, we need to run it as system services. If your operating +system uses Systemd, you can use Systemd ``service files`` to manage +both the Celery and Paste processes as described below. In the Systemd configuration below, MediaGoblin log files are kept in the ``/var/log/mediagoblin`` directory. Create the directory and give @@ -512,64 +512,65 @@ it the proper permissions:: Place the following service files in the ``/etc/systemd/system/`` directory. The first file should be named -``mediagoblin-celeryd.service``. Be sure to modify it to suit your +``mediagoblin-paster.service``. Be sure to modify it to suit your environment's setup: .. code-block:: bash # Set the WorkingDirectory and Environment values to match your environment. [Unit] - Description=MediaGoblin Celery - After=rabbitmq-server.service + Description=Mediagoblin [Service] + Type=simple User=mediagoblin Group=mediagoblin - Type=simple + Environment=CELERY_ALWAYS_EAGER=false WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin - Environment=MEDIAGOBLIN_CONFIG=/srv/mediagoblin.example.org/mediagoblin/mediagoblin.ini \ - CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery - ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/celery worker \ - --logfile=/var/log/mediagoblin/celery.log \ - --loglevel=INFO + ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ + /srv/mediagoblin.example.org/mediagoblin/paste.ini \ + --log-file=/var/log/mediagoblin/mediagoblin.log \ + --server-name=main [Install] WantedBy=multi-user.target - -The second file should be named ``mediagoblin-paster.service``: +The second file should be named ``mediagoblin-celeryd.service``: .. code-block:: bash # Set the WorkingDirectory and Environment values to match your environment. [Unit] - Description=Mediagoblin + Description=MediaGoblin Celery + After=rabbitmq-server.service [Service] - Type=simple User=mediagoblin Group=mediagoblin - Environment=CELERY_ALWAYS_EAGER=false + Type=simple WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin - ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ - /srv/mediagoblin.example.org/mediagoblin/paste.ini \ - --log-file=/var/log/mediagoblin/mediagoblin.log \ - --server-name=main + Environment=MEDIAGOBLIN_CONFIG=/srv/mediagoblin.example.org/mediagoblin/mediagoblin.ini \ + CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery + ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/celery worker \ + --logfile=/var/log/mediagoblin/celery.log \ + --loglevel=INFO [Install] WantedBy=multi-user.target +For details on this approach with a separate Celery process, see +:ref:`background-media-processing`. Enable these processes to start at boot by entering:: - sudo systemctl enable mediagoblin-celeryd.service && sudo systemctl enable mediagoblin-paster.service + sudo systemctl enable mediagoblin-paster.service + sudo systemctl enable mediagoblin-celeryd.service Start the processes for the current session with:: - sudo systemctl start mediagoblin-celeryd.service sudo systemctl start mediagoblin-paster.service - + sudo systemctl start mediagoblin-celeryd.service If either command above gives you an error, you can investigate the cause of the error by entering either of:: @@ -577,15 +578,8 @@ the error by entering either of:: sudo systemctl status mediagoblin-celeryd.service sudo systemctl status mediagoblin-paster.service - The above ``systemctl status`` command is also useful if you ever want to -confirm that a process is still running. If you make any changes to the ".service" -files, you can reload the service files and restart MediaGoblin by entering:: - - sudo systemctl daemon-reload - sudo systemctl restart mediagoblin-celeryd.service - sudo systemctl restart mediagoblin-paster.service - +confirm that a process is still running. Assuming the above was successful, you should now have a MediaGoblin server that will continue to operate, even after being restarted. @@ -602,6 +596,14 @@ To restart MediaGoblin after making configuration changes, run:: sudo systemctl restart mediagoblin-celeryd.service sudo systemctl restart mediagoblin-paster.service +If you make any changes to the ".service" files, you must first issue +a `daemon-reload` command to refresh Systemd and then restart +MediaGoblin with:: + + sudo systemctl daemon-reload + sudo systemctl restart mediagoblin-celeryd.service + sudo systemctl restart mediagoblin-paster.service + What next? ---------- diff --git a/docs/source/siteadmin/production-deployments.rst b/docs/source/siteadmin/production-deployments.rst index 02a7bd8f..2a78f479 100644 --- a/docs/source/siteadmin/production-deployments.rst +++ b/docs/source/siteadmin/production-deployments.rst @@ -11,13 +11,12 @@ Dedication along with this software. If not, see <http://creativecommons.org/publicdomain/zero/1.0/>. -========================================= -Considerations for Production Deployments -========================================= +================================================= +Further Considerations for Production Deployments +================================================= -This document contains a number of suggestions for deploying -MediaGoblin in actual production environments. Consider -":doc:`deploying`" for a basic overview of how to deploy MediaGoblin. +This page extends upon our ":doc:`deploying`" guide to describe some common +issues affecting production deployments. Should I Keep Open Registration Enabled? @@ -43,122 +42,64 @@ We hope to have a better solution to this situation shortly. We apologize for the inconvenience in the meanwhile. -Security Considerations ------------------------ +Confidential Files +------------------ .. warning:: - The directory ``user_dev/crypto/`` contains some very - sensitive files. - Especially the ``itsdangeroussecret.bin`` is very important - for session security. Make sure not to leak its contents anywhere. - If the contents gets leaked nevertheless, delete your file - and restart the server, so that it creates a new secret key. - All previous sessions will be invalidated. + The directory ``user_dev/crypto/`` contains confidential information. In + particular, the ``itsdangeroussecret.bin`` is important for the security of + login sessions. Make sure not to publish its contents anywhere. If the + contents gets leaked nevertheless, delete your file and restart the server, + so that it creates a new secret key. All previous login sessions will be + invalidated. -.. _init-script: +.. _background-media-processing: -Alternative init scripts ------------------------- +Background Media Processing +--------------------------- -If your system does not use Systemd, you can use the following command as the -basis for an init script: - -.. code-block:: bash - - CELERY_ALWAYS_EAGER=true \ - /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ - /srv/mediagoblin.example.org/mediagoblin/paste.ini \ - --pid-file=/var/run/mediagoblin.pid \ - --server-name=main - -The above configuration places MediaGoblin in "always eager" mode -with Celery, this means that submissions of content will be processed -synchronously, and the user will advance to the next page only after -processing is complete. If we take Celery out of "always eager mode," -the user will be able to immediately return to the MediaGoblin site -while processing is ongoing. In these cases, use the following command -as the basis for your script: - -.. code-block:: bash - - CELERY_ALWAYS_EAGER=false \ - /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ - /srv/mediagoblin.example.org/mediagoblin/paste.ini \ - --pid-file=/var/run/mediagoblin.pid \ - --server-name=main - - -Members of the MediaGoblin community have provided init scripts for the -following GNU/Linux distributions: - -Arch Linux - * `MediaGoblin - ArchLinux rc.d scripts - <http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts>`_ - by `Jeremy Pope <http://jpope.org/>`_ - * `Mediagoblin init script on Archlinux - <http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/>`_ - by `Chimo <http://chimo.chromic.org/>`_ - -You can reference these scripts to create an init script for your own operating -system. Similar scripts will be in your system's ``/etc/init.d/`` -or ``/etc/rc.d/`` directory, but the specifics of an init script will vary from -one distribution to the next. - -.. _separate-celery: - -Separate celery ---------------- - -":doc:`deploying`" describes a configuration with a separate Celery process, but -the following section covers this in more detail. +":doc:`deploying`" covers use of a separate Celery process, but this sections +describes this in more detail. MediaGoblin uses `Celery`_ to handle heavy and long-running tasks. Celery can be launched in two ways: -1. Embedded in the MediaGoblin WSGI application [#f-mediagoblin-wsgi-app]_. - This is the way ``./lazyserver.sh`` does it for you. It's simple as you - only have to run one process. The only bad thing with this is that the - heavy and long-running tasks will run *in* the webserver, keeping the user - waiting each time some heavy lifting is needed as in for example processing - a video. This could lead to problems as an aborted connection will halt any - processing and since most front-end web servers *will* terminate your - connection if it doesn't get any response from the MediaGoblin WSGI - application in a while. - -2. As a separate process communicating with the MediaGoblin WSGI application - via a `broker`_. This offloads the heavy lifting from the MediaGoblin WSGI - application and users will be able to continue to browse the site while the - media is being processed in the background. +1. **Embedded in the main MediaGoblin web application.** This is the way + ``./lazyserver.sh`` does it for you. It's simple as you only have to run one + process. The only bad thing with this is that the heavy and long-running + tasks will run *in* the webserver, keeping the user waiting each time some + heavy lifting is needed as in for example processing a video. This could lead + to problems as an aborted connection will halt any processing and since most + front-end web servers *will* terminate your connection if it doesn't get any + response from the MediaGoblin web application in a while. This approach is + suitable for development, small sites or when primarily using :doc:`command + line uploads <commandline-upload>`. + +2. **As a separate web application and media processing application + (recommended).** In this approach, the MediaGoblin web application delegates + all media processing to a task queue via a `broker`_ (task queue). This is + the approach used in our :doc:`deployment guide <deploying>`, with RabbitMQ + as the broker. This offloads the heavy lifting from the MediaGoblin web + application and users will be able to continue to browse the site while the + media is being processed in the background. This approach provided the best + user experience and is recommended for production sites. + +The choice between these two behaviours is controlled by the +``CELERY_ALWAYS_EAGER`` environment variable. Specifying ``true`` instructs +MediaGoblin to processing media within the web application while you wait. +Specifying ``false`` instructs MediaGoblin to use background processing. .. _`broker`: http://docs.celeryproject.org/en/latest/getting-started/brokers/ .. _`celery`: http://www.celeryproject.org/ -.. [#f-mediagoblin-wsgi-app] The MediaGoblin WSGI application is the part that - of MediaGoblin that processes HTTP requests. - -To launch Celery separately from the MediaGoblin WSGI application: - -1. Make sure that the ``CELERY_ALWAYS_EAGER`` environment variable is unset or - set to ``false`` when launching the MediaGoblin WSGI application. -2. Start the ``celeryd`` main process with - - .. code-block:: bash - - CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd - -If you use our example Systemd ``service files``, Celery will be set to the -"CELERY_ALWAYS_EAGER=false" value by default. This will provide your users -with the best user experience, as all media processing will be done in the -background. - .. _sentry: -Set up sentry to monitor exceptions ------------------------------------ +Error Monitoring with Sentry +---------------------------- We have a plugin for `raven`_ integration, see the ":doc:`/plugindocs/raven`" documentation. |