diff options
Diffstat (limited to 'docs/source/siteadmin/deploying.rst')
| -rw-r--r-- | docs/source/siteadmin/deploying.rst | 349 |
1 files changed, 232 insertions, 117 deletions
diff --git a/docs/source/siteadmin/deploying.rst b/docs/source/siteadmin/deploying.rst index 3f4a59cd..80b60642 100644 --- a/docs/source/siteadmin/deploying.rst +++ b/docs/source/siteadmin/deploying.rst @@ -17,25 +17,37 @@ Deploying MediaGoblin ===================== -GNU MediaGoblin is fairly new and so at the time of writing, there -aren't easy package-manager-friendly methods to install MediaGoblin. -However, doing a basic install isn't too complex in and of itself. +GNU MediaGoblin is fairly new, and so at the time of writing there aren't +easy package-manager-friendly methods to install it. However, doing a basic +install isn't too complex in and of itself. Following this deployment guide +will take you step-by-step through setting up your own instance of MediaGoblin. -There's an almost infinite way to deploy things... for now, we'll keep -it simple with some assumptions and use a setup that combines -mediagoblin + virtualenv + fastcgi + nginx on a .deb or .rpm based -GNU/Linux distro. +Of course, when it comes to setting up web applications like MediaGoblin, +there's an almost infinite way to deploy things, so for now, we'll keep it +simple with some assumptions. We recommend a setup that combines MediaGoblin + +virtualenv + FastCGI + Nginx on a .deb- or .rpm-based GNU/Linux distro. + +Other deployment options (e.g., deploying on FreeBSD, Arch Linux, using +Apache, etc.) are possible, though! If you'd prefer a different deployment +approach, see our +`Deployment wiki page <http://wiki.mediagoblin.org/Deployment>`_. .. note:: These tools are for site administrators wanting to deploy a fresh - install. If instead you want to join in as a contributor, see our + install. If you want to join in as a contributor, see our `Hacking HOWTO <http://wiki.mediagoblin.org/HackingHowto>`_ instead. - There are also many ways to install servers... for the sake of - simplicity, our instructions below describe installing with nginx. - For more recipes, including Apache, see - `our wiki <http://wiki.mediagoblin.org/Deployment>`_. +.. note:: + + Throughout the documentation we use the ``sudo`` command to indicate that + an instruction requires elevated user privileges to run. You can issue + these commands as the ``root`` user if you prefer. + + If you need help configuring ``sudo``, see the + `Debian wiki <https://wiki.debian.org/sudo/>`_ or the + `Fedora Project wiki <https://fedoraproject.org/wiki/Configuring_Sudo/>`_. + Prepare System -------------- @@ -45,25 +57,32 @@ Dependencies MediaGoblin has the following core dependencies: -- Python 2.6 or 2.7 +- Python 2.7 or Python 3.4+ - `python-lxml <http://lxml.de/>`_ - `git <http://git-scm.com/>`_ - `SQLite <http://www.sqlite.org/>`_/`PostgreSQL <http://www.postgresql.org/>`_ - `Python Imaging Library <http://www.pythonware.com/products/pil/>`_ (PIL) - `virtualenv <http://www.virtualenv.org/>`_ +- `nodejs <https://nodejs.org>`_ -On a DEB-based system (e.g Debian, gNewSense, Trisquel, Ubuntu, and +On a DEB-based system (e.g Debian, gNewSense, Trisquel, *buntu, and derivatives) issue the following command:: sudo apt-get install git-core python python-dev python-lxml \ - python-imaging python-virtualenv + python-imaging python-virtualenv npm nodejs-legacy automake \ + nginx On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the following command:: - yum install python-paste-deploy python-paste-script \ + sudo yum install python-paste-deploy python-paste-script \ git-core python python-devel python-lxml python-imaging \ - python-virtualenv + python-virtualenv npm automake nginx + +(Note: MediaGoblin now officially supports Python 3. You may instead +substitute from "python" to "python3" for most package names in the +Debian instructions and this should cover dependency installation. +These instructions have not yet been tested on Fedora.) Configure PostgreSQL ~~~~~~~~~~~~~~~~~~~~ @@ -75,31 +94,54 @@ Configure PostgreSQL For medium to large deployments we recommend PostgreSQL. - If you don't want/need postgres, skip this section. + If you don't want/need PostgreSQL, skip this section. -These are the packages needed for Debian Wheezy (stable):: +These are the packages needed for Debian Jessie (stable):: sudo apt-get install postgresql postgresql-client python-psycopg2 +These are the packages needed for an RPM-based system:: + + sudo yum install postgresql postgresql-server python-psycopg2 + +An rpm-based system also requires that you initialize and start the +PostgreSQL database with a few commands. The following commands are +not needed on a Debian-based platform, however:: + + sudo /usr/bin/postgresql-setup initdb + sudo systemctl enable postgresql + sudo systemctl start postgresql + The installation process will create a new *system* user named ``postgres``, -it will have privilegies sufficient to manage the database. We will create a -new database user with restricted privilegies and a new database owned by our +which will have privileges sufficient to manage the database. We will create a +new database user with restricted privileges and a new database owned by our restricted database user for our MediaGoblin instance. In this example, the database user will be ``mediagoblin`` and the database name will be ``mediagoblin`` too. -To create our new user, run:: +We'll add these entities by first switching to the *postgres* account:: + + sudo su - postgres - sudo -u postgres createuser -A -D mediagoblin +This will change your prompt to a shell prompt, such as *-bash-4.2$*. Enter +the following *createuser* and *createdb* commands at that prompt. We'll +create the *mediagoblin* database user first:: -then create the database all our MediaGoblin data should be stored in:: + # this command and the one that follows are run as the ``postgres`` user: + createuser -A -D mediagoblin - sudo -u postgres createdb -E UNICODE -O mediagoblin mediagoblin +Then we'll create the database where all of our MediaGoblin data will be stored:: + + createdb -E UNICODE -O mediagoblin mediagoblin where the first ``mediagoblin`` is the database owner and the second ``mediagoblin`` is the database name. +Type ``exit`` to exit from the 'postgres' user account.:: + + exit + .. caution:: Where is the password? These steps enable you to authenticate to the database in a password-less @@ -118,35 +160,37 @@ Drop Privileges for MediaGoblin MediaGoblin does not require special permissions or elevated access to run. As such, the preferred way to run MediaGoblin is to create a dedicated, unprivileged system user for the sole purpose of running -MediaGoblin. Running MediaGoblin processes under an unpriviledged system user +MediaGoblin. Running MediaGoblin processes under an unprivileged system user helps to keep it more secure. The following command (entered as root or with sudo) will create a system account with a username of ``mediagoblin``. You may choose a different -username if you wish.:: - - adduser --system mediagoblin +username if you wish. -No password will be assigned to this account, and you will not be able -to log in as this user. To switch to this account, enter either:: +If you are using a Debian-based system, enter this command:: - sudo -u mediagoblin /bin/bash # (if you have sudo permissions) + sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g www-data mediagoblin -or:: +If you are using an RPM-based system, enter this command:: - su mediagoblin -s /bin/bash # (if you have to use root permissions) + sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g nginx mediagoblin -You may get a warning similar to this when entering these commands:: +This will create a ``mediagoblin`` user and assign it to a group that is +associated with the web server. This will ensure that the web server can +read the media files (images, videos, etc.) that users upload. - warning: cannot change directory to /home/mediagoblin: No such file or directory - -You can disregard this warning. To return to your regular user account after -using the system account, just enter ``exit``. +We will also create a ``mediagoblin`` group and associate the mediagoblin +user with that group, as well:: + + sudo groupadd mediagoblin && sudo usermod --append -G mediagoblin mediagoblin + +No password will be assigned to this account, and you will not be able +to log in as this user. To switch to this account, enter:: -.. note:: + sudo su mediagoblin -s /bin/bash - Unless otherwise noted, the remainder of this document assumes that all - operations are performed using this unpriviledged account. +To return to your regular user account after using the system account, type +``exit``. .. _create-mediagoblin-directory: @@ -156,73 +200,74 @@ Create a MediaGoblin Directory You should create a working directory for MediaGoblin. This document assumes your local git repository will be located at ``/srv/mediagoblin.example.org/mediagoblin/``. -Substitute your prefered local deployment path as needed. +Substitute your preferred local deployment path as needed. Setting up the working directory requires that we first create the directory -with elevated priviledges, and then assign ownership of the directory -to the unpriviledged system account. +with elevated privileges, and then assign ownership of the directory +to the unprivileged system account. -To do this, enter either of the following commands, changing the defaults -to suit your particular requirements:: +To do this, enter the following command, changing the defaults to suit your +particular requirements. On a Debian-based platform you will enter this:: - sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:mediagoblin /srv/mediagoblin.example.org + sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:www-data /srv/mediagoblin.example.org -or (as the root user):: +On an RPM-based distribution, enter this command:: - mkdir -p /srv/mediagoblin.example.org && chown -hR mediagoblin:mediagoblin /srv/mediagoblin.example.org + sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:nginx /srv/mediagoblin.example.org +.. note:: -Install MediaGoblin and Virtualenv ----------------------------------- + Unless otherwise noted, the remainder of this document assumes that all + operations are performed using this unprivileged account. -.. note:: - MediaGoblin is still developing rapidly. As a result - the following instructions recommend installing from the ``master`` - branch of the git repository. Eventually production deployments will - want to transition to running from more consistent releases. +Install MediaGoblin and Virtualenv +---------------------------------- -We will now clone the MediaGoblin source code repository and setup and -configure the necessary services. Modify these commands to -suit your own environment. As a reminder, you should enter these -commands using your unpriviledged system account. +We will now switch to our 'mediagoblin' system account, and then set up +our MediaGoblin source code repository and its necessary services. +You should modify these commands to suit your own environment. Change to the MediaGoblin directory that you just created:: - cd /srv/mediagoblin.example.org + sudo su mediagoblin -s /bin/bash # to change to the 'mediagoblin' account + $ cd /srv/mediagoblin.example.org Clone the MediaGoblin repository and set up the git submodules:: - git clone git://gitorious.org/mediagoblin/mediagoblin.git - cd mediagoblin - git submodule init && git submodule update + $ git clone git://git.savannah.gnu.org/mediagoblin.git -b stable + $ cd mediagoblin + $ git submodule init && git submodule update +.. note:: -And set up the in-package virtualenv:: + The MediaGoblin repository used to be on gitorious.org, but since + gitorious.org shut down, we had to move. We are presently on + Savannah. You may need to update your git repository location:: - (virtualenv --system-site-packages . || virtualenv .) && ./bin/python setup.py develop + $ git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git -.. note:: +Set up the hacking environment:: + + $ ./bootstrap.sh && ./configure && make - We presently have an **experimental** make-style deployment system. if - you'd like to try it, instead of the above command, you can run:: +(Note that if you'd prefer to run MediaGoblin with Python 3, pass in +`--with-python3` to the `./configure` command.) - ./experimental-bootstrap.sh && ./configure && make +Create and set the proper permissions on the ``user_dev`` directory. +This directory will be used to store uploaded media files:: - This also includes a number of nice features, such as keeping your - viratualenv up to date by simply running `make update`. + $ mkdir user_dev && chmod 750 user_dev - Note: this is liable to break. Use this method with caution. +Assuming you are going to deploy with FastCGI, you should also install +flup:: -.. :: + $ ./bin/easy_install flup - (NOTE: Is this still relevant?) +(Note, if you're running Python 2, which you probably are at this +point in MediaGoblin's development, you'll need to run:) - If you have problems here, consider trying to install virtualenv - with the ``--distribute`` or ``--no-site-packages`` options. If - your system's default Python is in the 3.x series you may need to - run ``virtualenv`` with the ``--python=python2.7`` or - ``--python=python2.6`` options. + $ ./bin/easy_install flup==1.0.3.dev-20110405 The above provides an in-package install of ``virtualenv``. While this is counter to the conventional ``virtualenv`` configuration, it is @@ -230,26 +275,36 @@ more reliable and considerably easier to configure and illustrate. If you're familiar with Python packaging you may consider deploying with your preferred method. -Assuming you are going to deploy with FastCGI, you should also install -flup:: - - ./bin/easy_install flup - -(Sometimes this breaks because flup's site is flakey. If it does for -you, try):: - - ./bin/easy_install https://pypi.python.org/pypi/flup/1.0.3.dev-20110405 +.. note:: -This concludes the initial configuration of the development + What if you don't want an in-package ``virtualenv``? Maybe you + have your own ``virtualenv``, or you are building a MediaGoblin + package for a distribution. There's no need necessarily for the + virtualenv produced by ``./configure && make`` by default other + than attempting to simplify work for developers and people + deploying by hiding all the virtualenv and bower complexity. + + If you want to install all of MediaGoblin's libraries + independently, that's totally fine! You can pass the flag + ``--without-virtualenv`` which will skip this step. + But you will need to install all those libraries manually and make + sure they are on your ``PYTHONPATH`` yourself! (You can still use + ``python setup.py develop`` to install some of those libraries, + but note that no ``./bin/python`` will be set up for you via this + method, since no virtualenv is set up for you!) + +This concludes the initial configuration of the MediaGoblin environment. In the future, when you update your codebase, you should also run:: - git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate + $ git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate -Note: If you are running an active site, depending on your server -configuration, you may need to stop it first or the dbupdate command -may hang (and it's certainly a good idea to restart it after the -update) +.. note:: + + Note: If you are running an active site, depending on your server + configuration, you may need to stop it first or the dbupdate command + may hang (and it's certainly a good idea to restart it after the + update) Deploy MediaGoblin Services @@ -259,23 +314,24 @@ Edit site configuration ~~~~~~~~~~~~~~~~~~~~~~~ A few basic properties must be set before MediaGoblin will work. First -make a copy of ``mediagoblin.ini`` for editing so the original config -file isn't lost:: +make a copy of ``mediagoblin.ini`` and ``paste.ini`` for editing so the original +config files aren't lost (you likely won't need to edit the paste configuration, +but we'll make a local copy of it just in case):: - cp mediagoblin.ini mediagoblin_local.ini + $ cp -av mediagoblin.ini mediagoblin_local.ini && cp -av paste.ini paste_local.ini -Then: +Then edit mediagoblin_local.ini: - Set ``email_sender_address`` to the address you wish to be used as the sender for system-generated emails - Edit ``direct_remote_path``, ``base_dir``, and ``base_url`` if your mediagoblin directory is not the root directory of your - vhost. + site. Configure MediaGoblin to use the PostgreSQL database ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If you are using postgres, edit the ``[mediagoblin]`` section in your +If you are using PostgreSQL, edit the ``[mediagoblin]`` section in your ``mediagoblin_local.ini`` and put in:: sql_engine = postgresql:///mediagoblin @@ -289,7 +345,7 @@ Update database data structures Before you start using the database, you need to run:: - ./bin/gmg dbupdate + $ ./bin/gmg dbupdate to populate the database with the MediaGoblin data structures. @@ -300,20 +356,25 @@ Test the Server At this point MediaGoblin should be properly installed. You can test the deployment with the following command:: - ./lazyserver.sh --server-name=broadcast + $ ./lazyserver.sh --server-name=broadcast You should be able to connect to the machine on port 6543 in your browser to confirm that the service is operable. +The next series of commands will need to be run as a privileged user. Type +exit to return to the root/sudo account.:: + + exit + .. _webserver-config: FastCGI and nginx ~~~~~~~~~~~~~~~~~ -This configuration example will use nginx, however, you may +This configuration example will use Nginx, however, you may use any webserver of your choice as long as it supports the FastCGI -protocol. If you do not already have a web server, consider nginx, as +protocol. If you do not already have a web server, consider Nginx, as the configuration files may be more clear than the alternatives. @@ -321,13 +382,22 @@ Create a configuration file at ``/srv/mediagoblin.example.org/nginx.conf`` and create a symbolic link into a directory that will be included in your ``nginx`` configuration (e.g. "``/etc/nginx/sites-enabled`` or ``/etc/nginx/conf.d``) with -one of the following commands (as the root user):: +one of the following commands. - ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/ - ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/ +On a DEB-based system (e.g Debian, gNewSense, Trisquel, *buntu, and +derivatives) issue the following commands:: + + sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/ + sudo systemctl enable nginx + +On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the +following commands:: -Modify these commands and locations depending on your preferences and -the existing configuration of your nginx instance. The contents of + sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/ + sudo systemctl enable nginx + +You can modify these commands and locations depending on your preferences and +the existing configuration of your Nginx instance. The contents of this ``nginx.conf`` file should be modeled on the following:: server { @@ -344,7 +414,7 @@ this ``nginx.conf`` file should be modeled on the following:: gzip on; gzip_min_length 1024; gzip_buffers 4 32k; - gzip_types text/plain text/html application/x-javascript text/javascript text/xml text/css; + gzip_types text/plain application/x-javascript text/javascript text/xml text/css; ##################################### # Mounting MediaGoblin stuff @@ -387,25 +457,43 @@ this ``nginx.conf`` file should be modeled on the following:: fastcgi_pass 127.0.0.1:26543; include /etc/nginx/fastcgi_params; - # our understanding vs nginx's handling of script_name vs + # our understanding vs Nginx's handling of script_name vs # path_info don't match :) fastcgi_param PATH_INFO $fastcgi_script_name; fastcgi_param SCRIPT_NAME ""; } } -Now, nginx instance is configured to serve the MediaGoblin -application. Perform a quick test to ensure that this configuration -works. Restart nginx so it picks up your changes, with a command that -resembles one of the following (as the root user):: +The first four ``location`` directives instruct Nginx to serve the +static and uploaded files directly rather than through the MediaGoblin +process. This approach is faster and requires less memory. + +.. note:: + + The user who owns the Nginx process, normally ``www-data`` or ``nginx``, + requires execute permission on the directories ``static``, + ``public``, ``theme_static`` and ``plugin_static`` plus all their + parent directories. This user also requires read permission on all + the files within these directories. This is normally the default. + +Nginx is now configured to serve the MediaGoblin application. Perform a quick +test to ensure that this configuration works:: + + nginx -t + +If you encounter any errors, review your Nginx configuration files, and try to +resolve them. If you do not encounter any errors, you can start your Nginx +server with one of the following commands (depending on your environment):: sudo /etc/init.d/nginx restart sudo /etc/rc.d/nginx restart + sudo systemctl restart nginx Now start MediaGoblin. Use the following command sequence as an example:: cd /srv/mediagoblin.example.org/mediagoblin/ + su mediagoblin -s /bin/bash ./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543 Visit the site you've set up in your browser by visiting @@ -426,6 +514,29 @@ Instructions and scripts for running MediaGoblin on an Apache server can be found on the `MediaGoblin wiki <http://wiki.mediagoblin.org/Deployment>`_. +Should I Keep Open Registration Enabled? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Unfortunately, in this current release of MediaGoblin we are suffering +from spammers registering to public instances en masse. As such, you +may want to either: + +a) Disable registration on your instance and just make + accounts for people you know and trust (eg via the `gmg adduser` + command). You can disable registration in your mediagoblin.ini + like so:: + + [mediagoblin] + allow_registration = false + +b) Enable a CAPTCHA plugin. But unfortunately, though some CAPTCHA + plugins exist, for various reasons we do not have any general + recommendations we can make at this point. + +We hope to have a better solution to this situation shortly. We +apologize for the inconvenience in the meanwhile. + + Security Considerations ~~~~~~~~~~~~~~~~~~~~~~~ @@ -439,3 +550,7 @@ Security Considerations and restart the server, so that it creates a new secret key. All previous sessions will be invalidated. +.. + Local variables: + fill-column: 70 + End: |