diff options
Diffstat (limited to 'docs/source/siteadmin')
| -rw-r--r-- | docs/source/siteadmin/commandline-upload.rst | 18 | ||||
| -rw-r--r-- | docs/source/siteadmin/configuration.rst | 32 | ||||
| -rw-r--r-- | docs/source/siteadmin/deploying.rst | 349 | ||||
| -rw-r--r-- | docs/source/siteadmin/foreword.rst | 9 | ||||
| -rw-r--r-- | docs/source/siteadmin/media-types.rst | 115 | ||||
| -rw-r--r-- | docs/source/siteadmin/plugins.rst | 29 | ||||
| -rw-r--r-- | docs/source/siteadmin/production-deployments.rst | 195 | ||||
| -rw-r--r-- | docs/source/siteadmin/relnotes.rst | 348 | ||||
| -rw-r--r-- | docs/source/siteadmin/theming.rst | 16 |
9 files changed, 873 insertions, 238 deletions
diff --git a/docs/source/siteadmin/commandline-upload.rst b/docs/source/siteadmin/commandline-upload.rst index 15b2377d..ef597a44 100644 --- a/docs/source/siteadmin/commandline-upload.rst +++ b/docs/source/siteadmin/commandline-upload.rst @@ -15,7 +15,13 @@ Command-line uploading ====================== -Want to submit media via the command line? It's fairly easy to do:: +If you're a site administrator and have access to the server then you +can use the 'addmedia' task. If you're just a user and want to upload +media by the command line you can. This can be done with the pump.io +API. There is `p <https://github.com/xray7224/p/>`_, which will allow you +to easily upload media from the command line, follow p's docs to do that. + +To use the addmedia command:: ./bin/gmg addmedia username your_media.jpg @@ -60,14 +66,14 @@ The above is an example of a very simple metadata.csv file. The batchaddmedia script would read this and attempt to upload only two pieces of media, and would be able to automatically name them appropriately. -The csv file +The CSV file ============ The location column ------------------- The location column is the one column that is absolutely necessary for uploading your media. This gives a path to each piece of media you upload. This can either a path to a local file or a direct link to remote media (with the -link in http format). As you can see in the example above the (fake) media was +link in HTTP format). As you can see in the example above the (fake) media was stored remotely on "www.example.net". Other internal nodes @@ -78,8 +84,8 @@ provide default information for your media entry, but as you'll see below, it's just as easy to provide this information through the correct metadata columns. - **id** is used to identify the media entry to the user in case of an error in the batchaddmedia script. -- **license** is used to set a license for your piece a media for mediagoblin's use. This must be a URI. -- **title** will set the title displayed to mediagoblin users. +- **license** is used to set a license for your piece a media for MediaGoblin's use. This must be a URI. +- **title** will set the title displayed to MediaGoblin users. - **description** will set a description of your media. Metadata columns @@ -101,7 +107,7 @@ information of uploaded media entries. - **dc:title** sets a title for your media entry. - **dc:description** sets a description of your media entry. -If both a metadata column and an internal node for the title are provided, mediagoblin +If both a metadata column and an internal node for the title are provided, MediaGoblin will use the internal node as the media entry's display name. This makes it so that if you want to display a piece of media with a different title than the one provided in its metadata, you can just provide different data for diff --git a/docs/source/siteadmin/configuration.rst b/docs/source/siteadmin/configuration.rst index 3da5cdd9..6b8cd225 100644 --- a/docs/source/siteadmin/configuration.rst +++ b/docs/source/siteadmin/configuration.rst @@ -39,12 +39,12 @@ paste.ini <http://pythonpaste.org/script/>`_). It also sets up some middleware that you can mostly ignore, except to configure sessions... more on that later. If you are adding a different - Python server other than fastcgi / plain HTTP, you might configure + Python server other than FastCGI / plain HTTP, you might configure it here. You probably won't need to change this file very much. There's one more file that you certainly won't change unless you're -making coding contributions to mediagoblin, but which can be useful to +making coding contributions to MediaGoblin, but which can be useful to read and reference: mediagoblin/config_spec.ini @@ -109,6 +109,34 @@ they sound like. - email_smtp_user - email_smtp_pass +Changing data directory +----------------------- + +MediaGoblin by default stores your data in wherever ``data_basedir``. +This can be changed by changing the value in your ``mediagoblin.ini`` file +for example:: + + [DEFAULT] + data_basedir = "/var/mediagoblin/user_data" + +For efficiency reasons MediaGoblin doesn't serve these files itself and +instead leaves that to the webserver. You will have to alter the location +to match the path in ``data_basedir``. + +If you use ``lazyserver.sh`` you need to change the ``paste.ini`` file:: + + [app:mediagoblin] + /mgoblin_media = /var/mediagoblin/user_data + +If you use Nginx you need to change the config:: + + # Instance specific media: + location /mgoblin_media/ { + alias /var/mediagoblin/user_data; + } + +Once you have done this you will need to move any existing media you had in the +old directory to the new directory so existing media still can be displayed. All other configuration changes ------------------------------- diff --git a/docs/source/siteadmin/deploying.rst b/docs/source/siteadmin/deploying.rst index 3f4a59cd..80b60642 100644 --- a/docs/source/siteadmin/deploying.rst +++ b/docs/source/siteadmin/deploying.rst @@ -17,25 +17,37 @@ Deploying MediaGoblin ===================== -GNU MediaGoblin is fairly new and so at the time of writing, there -aren't easy package-manager-friendly methods to install MediaGoblin. -However, doing a basic install isn't too complex in and of itself. +GNU MediaGoblin is fairly new, and so at the time of writing there aren't +easy package-manager-friendly methods to install it. However, doing a basic +install isn't too complex in and of itself. Following this deployment guide +will take you step-by-step through setting up your own instance of MediaGoblin. -There's an almost infinite way to deploy things... for now, we'll keep -it simple with some assumptions and use a setup that combines -mediagoblin + virtualenv + fastcgi + nginx on a .deb or .rpm based -GNU/Linux distro. +Of course, when it comes to setting up web applications like MediaGoblin, +there's an almost infinite way to deploy things, so for now, we'll keep it +simple with some assumptions. We recommend a setup that combines MediaGoblin + +virtualenv + FastCGI + Nginx on a .deb- or .rpm-based GNU/Linux distro. + +Other deployment options (e.g., deploying on FreeBSD, Arch Linux, using +Apache, etc.) are possible, though! If you'd prefer a different deployment +approach, see our +`Deployment wiki page <http://wiki.mediagoblin.org/Deployment>`_. .. note:: These tools are for site administrators wanting to deploy a fresh - install. If instead you want to join in as a contributor, see our + install. If you want to join in as a contributor, see our `Hacking HOWTO <http://wiki.mediagoblin.org/HackingHowto>`_ instead. - There are also many ways to install servers... for the sake of - simplicity, our instructions below describe installing with nginx. - For more recipes, including Apache, see - `our wiki <http://wiki.mediagoblin.org/Deployment>`_. +.. note:: + + Throughout the documentation we use the ``sudo`` command to indicate that + an instruction requires elevated user privileges to run. You can issue + these commands as the ``root`` user if you prefer. + + If you need help configuring ``sudo``, see the + `Debian wiki <https://wiki.debian.org/sudo/>`_ or the + `Fedora Project wiki <https://fedoraproject.org/wiki/Configuring_Sudo/>`_. + Prepare System -------------- @@ -45,25 +57,32 @@ Dependencies MediaGoblin has the following core dependencies: -- Python 2.6 or 2.7 +- Python 2.7 or Python 3.4+ - `python-lxml <http://lxml.de/>`_ - `git <http://git-scm.com/>`_ - `SQLite <http://www.sqlite.org/>`_/`PostgreSQL <http://www.postgresql.org/>`_ - `Python Imaging Library <http://www.pythonware.com/products/pil/>`_ (PIL) - `virtualenv <http://www.virtualenv.org/>`_ +- `nodejs <https://nodejs.org>`_ -On a DEB-based system (e.g Debian, gNewSense, Trisquel, Ubuntu, and +On a DEB-based system (e.g Debian, gNewSense, Trisquel, *buntu, and derivatives) issue the following command:: sudo apt-get install git-core python python-dev python-lxml \ - python-imaging python-virtualenv + python-imaging python-virtualenv npm nodejs-legacy automake \ + nginx On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the following command:: - yum install python-paste-deploy python-paste-script \ + sudo yum install python-paste-deploy python-paste-script \ git-core python python-devel python-lxml python-imaging \ - python-virtualenv + python-virtualenv npm automake nginx + +(Note: MediaGoblin now officially supports Python 3. You may instead +substitute from "python" to "python3" for most package names in the +Debian instructions and this should cover dependency installation. +These instructions have not yet been tested on Fedora.) Configure PostgreSQL ~~~~~~~~~~~~~~~~~~~~ @@ -75,31 +94,54 @@ Configure PostgreSQL For medium to large deployments we recommend PostgreSQL. - If you don't want/need postgres, skip this section. + If you don't want/need PostgreSQL, skip this section. -These are the packages needed for Debian Wheezy (stable):: +These are the packages needed for Debian Jessie (stable):: sudo apt-get install postgresql postgresql-client python-psycopg2 +These are the packages needed for an RPM-based system:: + + sudo yum install postgresql postgresql-server python-psycopg2 + +An rpm-based system also requires that you initialize and start the +PostgreSQL database with a few commands. The following commands are +not needed on a Debian-based platform, however:: + + sudo /usr/bin/postgresql-setup initdb + sudo systemctl enable postgresql + sudo systemctl start postgresql + The installation process will create a new *system* user named ``postgres``, -it will have privilegies sufficient to manage the database. We will create a -new database user with restricted privilegies and a new database owned by our +which will have privileges sufficient to manage the database. We will create a +new database user with restricted privileges and a new database owned by our restricted database user for our MediaGoblin instance. In this example, the database user will be ``mediagoblin`` and the database name will be ``mediagoblin`` too. -To create our new user, run:: +We'll add these entities by first switching to the *postgres* account:: + + sudo su - postgres - sudo -u postgres createuser -A -D mediagoblin +This will change your prompt to a shell prompt, such as *-bash-4.2$*. Enter +the following *createuser* and *createdb* commands at that prompt. We'll +create the *mediagoblin* database user first:: -then create the database all our MediaGoblin data should be stored in:: + # this command and the one that follows are run as the ``postgres`` user: + createuser -A -D mediagoblin - sudo -u postgres createdb -E UNICODE -O mediagoblin mediagoblin +Then we'll create the database where all of our MediaGoblin data will be stored:: + + createdb -E UNICODE -O mediagoblin mediagoblin where the first ``mediagoblin`` is the database owner and the second ``mediagoblin`` is the database name. +Type ``exit`` to exit from the 'postgres' user account.:: + + exit + .. caution:: Where is the password? These steps enable you to authenticate to the database in a password-less @@ -118,35 +160,37 @@ Drop Privileges for MediaGoblin MediaGoblin does not require special permissions or elevated access to run. As such, the preferred way to run MediaGoblin is to create a dedicated, unprivileged system user for the sole purpose of running -MediaGoblin. Running MediaGoblin processes under an unpriviledged system user +MediaGoblin. Running MediaGoblin processes under an unprivileged system user helps to keep it more secure. The following command (entered as root or with sudo) will create a system account with a username of ``mediagoblin``. You may choose a different -username if you wish.:: - - adduser --system mediagoblin +username if you wish. -No password will be assigned to this account, and you will not be able -to log in as this user. To switch to this account, enter either:: +If you are using a Debian-based system, enter this command:: - sudo -u mediagoblin /bin/bash # (if you have sudo permissions) + sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g www-data mediagoblin -or:: +If you are using an RPM-based system, enter this command:: - su mediagoblin -s /bin/bash # (if you have to use root permissions) + sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g nginx mediagoblin -You may get a warning similar to this when entering these commands:: +This will create a ``mediagoblin`` user and assign it to a group that is +associated with the web server. This will ensure that the web server can +read the media files (images, videos, etc.) that users upload. - warning: cannot change directory to /home/mediagoblin: No such file or directory - -You can disregard this warning. To return to your regular user account after -using the system account, just enter ``exit``. +We will also create a ``mediagoblin`` group and associate the mediagoblin +user with that group, as well:: + + sudo groupadd mediagoblin && sudo usermod --append -G mediagoblin mediagoblin + +No password will be assigned to this account, and you will not be able +to log in as this user. To switch to this account, enter:: -.. note:: + sudo su mediagoblin -s /bin/bash - Unless otherwise noted, the remainder of this document assumes that all - operations are performed using this unpriviledged account. +To return to your regular user account after using the system account, type +``exit``. .. _create-mediagoblin-directory: @@ -156,73 +200,74 @@ Create a MediaGoblin Directory You should create a working directory for MediaGoblin. This document assumes your local git repository will be located at ``/srv/mediagoblin.example.org/mediagoblin/``. -Substitute your prefered local deployment path as needed. +Substitute your preferred local deployment path as needed. Setting up the working directory requires that we first create the directory -with elevated priviledges, and then assign ownership of the directory -to the unpriviledged system account. +with elevated privileges, and then assign ownership of the directory +to the unprivileged system account. -To do this, enter either of the following commands, changing the defaults -to suit your particular requirements:: +To do this, enter the following command, changing the defaults to suit your +particular requirements. On a Debian-based platform you will enter this:: - sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:mediagoblin /srv/mediagoblin.example.org + sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:www-data /srv/mediagoblin.example.org -or (as the root user):: +On an RPM-based distribution, enter this command:: - mkdir -p /srv/mediagoblin.example.org && chown -hR mediagoblin:mediagoblin /srv/mediagoblin.example.org + sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:nginx /srv/mediagoblin.example.org +.. note:: -Install MediaGoblin and Virtualenv ----------------------------------- + Unless otherwise noted, the remainder of this document assumes that all + operations are performed using this unprivileged account. -.. note:: - MediaGoblin is still developing rapidly. As a result - the following instructions recommend installing from the ``master`` - branch of the git repository. Eventually production deployments will - want to transition to running from more consistent releases. +Install MediaGoblin and Virtualenv +---------------------------------- -We will now clone the MediaGoblin source code repository and setup and -configure the necessary services. Modify these commands to -suit your own environment. As a reminder, you should enter these -commands using your unpriviledged system account. +We will now switch to our 'mediagoblin' system account, and then set up +our MediaGoblin source code repository and its necessary services. +You should modify these commands to suit your own environment. Change to the MediaGoblin directory that you just created:: - cd /srv/mediagoblin.example.org + sudo su mediagoblin -s /bin/bash # to change to the 'mediagoblin' account + $ cd /srv/mediagoblin.example.org Clone the MediaGoblin repository and set up the git submodules:: - git clone git://gitorious.org/mediagoblin/mediagoblin.git - cd mediagoblin - git submodule init && git submodule update + $ git clone git://git.savannah.gnu.org/mediagoblin.git -b stable + $ cd mediagoblin + $ git submodule init && git submodule update +.. note:: -And set up the in-package virtualenv:: + The MediaGoblin repository used to be on gitorious.org, but since + gitorious.org shut down, we had to move. We are presently on + Savannah. You may need to update your git repository location:: - (virtualenv --system-site-packages . || virtualenv .) && ./bin/python setup.py develop + $ git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git -.. note:: +Set up the hacking environment:: + + $ ./bootstrap.sh && ./configure && make - We presently have an **experimental** make-style deployment system. if - you'd like to try it, instead of the above command, you can run:: +(Note that if you'd prefer to run MediaGoblin with Python 3, pass in +`--with-python3` to the `./configure` command.) - ./experimental-bootstrap.sh && ./configure && make +Create and set the proper permissions on the ``user_dev`` directory. +This directory will be used to store uploaded media files:: - This also includes a number of nice features, such as keeping your - viratualenv up to date by simply running `make update`. + $ mkdir user_dev && chmod 750 user_dev - Note: this is liable to break. Use this method with caution. +Assuming you are going to deploy with FastCGI, you should also install +flup:: -.. :: + $ ./bin/easy_install flup - (NOTE: Is this still relevant?) +(Note, if you're running Python 2, which you probably are at this +point in MediaGoblin's development, you'll need to run:) - If you have problems here, consider trying to install virtualenv - with the ``--distribute`` or ``--no-site-packages`` options. If - your system's default Python is in the 3.x series you may need to - run ``virtualenv`` with the ``--python=python2.7`` or - ``--python=python2.6`` options. + $ ./bin/easy_install flup==1.0.3.dev-20110405 The above provides an in-package install of ``virtualenv``. While this is counter to the conventional ``virtualenv`` configuration, it is @@ -230,26 +275,36 @@ more reliable and considerably easier to configure and illustrate. If you're familiar with Python packaging you may consider deploying with your preferred method. -Assuming you are going to deploy with FastCGI, you should also install -flup:: - - ./bin/easy_install flup - -(Sometimes this breaks because flup's site is flakey. If it does for -you, try):: - - ./bin/easy_install https://pypi.python.org/pypi/flup/1.0.3.dev-20110405 +.. note:: -This concludes the initial configuration of the development + What if you don't want an in-package ``virtualenv``? Maybe you + have your own ``virtualenv``, or you are building a MediaGoblin + package for a distribution. There's no need necessarily for the + virtualenv produced by ``./configure && make`` by default other + than attempting to simplify work for developers and people + deploying by hiding all the virtualenv and bower complexity. + + If you want to install all of MediaGoblin's libraries + independently, that's totally fine! You can pass the flag + ``--without-virtualenv`` which will skip this step. + But you will need to install all those libraries manually and make + sure they are on your ``PYTHONPATH`` yourself! (You can still use + ``python setup.py develop`` to install some of those libraries, + but note that no ``./bin/python`` will be set up for you via this + method, since no virtualenv is set up for you!) + +This concludes the initial configuration of the MediaGoblin environment. In the future, when you update your codebase, you should also run:: - git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate + $ git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate -Note: If you are running an active site, depending on your server -configuration, you may need to stop it first or the dbupdate command -may hang (and it's certainly a good idea to restart it after the -update) +.. note:: + + Note: If you are running an active site, depending on your server + configuration, you may need to stop it first or the dbupdate command + may hang (and it's certainly a good idea to restart it after the + update) Deploy MediaGoblin Services @@ -259,23 +314,24 @@ Edit site configuration ~~~~~~~~~~~~~~~~~~~~~~~ A few basic properties must be set before MediaGoblin will work. First -make a copy of ``mediagoblin.ini`` for editing so the original config -file isn't lost:: +make a copy of ``mediagoblin.ini`` and ``paste.ini`` for editing so the original +config files aren't lost (you likely won't need to edit the paste configuration, +but we'll make a local copy of it just in case):: - cp mediagoblin.ini mediagoblin_local.ini + $ cp -av mediagoblin.ini mediagoblin_local.ini && cp -av paste.ini paste_local.ini -Then: +Then edit mediagoblin_local.ini: - Set ``email_sender_address`` to the address you wish to be used as the sender for system-generated emails - Edit ``direct_remote_path``, ``base_dir``, and ``base_url`` if your mediagoblin directory is not the root directory of your - vhost. + site. Configure MediaGoblin to use the PostgreSQL database ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If you are using postgres, edit the ``[mediagoblin]`` section in your +If you are using PostgreSQL, edit the ``[mediagoblin]`` section in your ``mediagoblin_local.ini`` and put in:: sql_engine = postgresql:///mediagoblin @@ -289,7 +345,7 @@ Update database data structures Before you start using the database, you need to run:: - ./bin/gmg dbupdate + $ ./bin/gmg dbupdate to populate the database with the MediaGoblin data structures. @@ -300,20 +356,25 @@ Test the Server At this point MediaGoblin should be properly installed. You can test the deployment with the following command:: - ./lazyserver.sh --server-name=broadcast + $ ./lazyserver.sh --server-name=broadcast You should be able to connect to the machine on port 6543 in your browser to confirm that the service is operable. +The next series of commands will need to be run as a privileged user. Type +exit to return to the root/sudo account.:: + + exit + .. _webserver-config: FastCGI and nginx ~~~~~~~~~~~~~~~~~ -This configuration example will use nginx, however, you may +This configuration example will use Nginx, however, you may use any webserver of your choice as long as it supports the FastCGI -protocol. If you do not already have a web server, consider nginx, as +protocol. If you do not already have a web server, consider Nginx, as the configuration files may be more clear than the alternatives. @@ -321,13 +382,22 @@ Create a configuration file at ``/srv/mediagoblin.example.org/nginx.conf`` and create a symbolic link into a directory that will be included in your ``nginx`` configuration (e.g. "``/etc/nginx/sites-enabled`` or ``/etc/nginx/conf.d``) with -one of the following commands (as the root user):: +one of the following commands. - ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/ - ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/ +On a DEB-based system (e.g Debian, gNewSense, Trisquel, *buntu, and +derivatives) issue the following commands:: + + sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/ + sudo systemctl enable nginx + +On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the +following commands:: -Modify these commands and locations depending on your preferences and -the existing configuration of your nginx instance. The contents of + sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/ + sudo systemctl enable nginx + +You can modify these commands and locations depending on your preferences and +the existing configuration of your Nginx instance. The contents of this ``nginx.conf`` file should be modeled on the following:: server { @@ -344,7 +414,7 @@ this ``nginx.conf`` file should be modeled on the following:: gzip on; gzip_min_length 1024; gzip_buffers 4 32k; - gzip_types text/plain text/html application/x-javascript text/javascript text/xml text/css; + gzip_types text/plain application/x-javascript text/javascript text/xml text/css; ##################################### # Mounting MediaGoblin stuff @@ -387,25 +457,43 @@ this ``nginx.conf`` file should be modeled on the following:: fastcgi_pass 127.0.0.1:26543; include /etc/nginx/fastcgi_params; - # our understanding vs nginx's handling of script_name vs + # our understanding vs Nginx's handling of script_name vs # path_info don't match :) fastcgi_param PATH_INFO $fastcgi_script_name; fastcgi_param SCRIPT_NAME ""; } } -Now, nginx instance is configured to serve the MediaGoblin -application. Perform a quick test to ensure that this configuration -works. Restart nginx so it picks up your changes, with a command that -resembles one of the following (as the root user):: +The first four ``location`` directives instruct Nginx to serve the +static and uploaded files directly rather than through the MediaGoblin +process. This approach is faster and requires less memory. + +.. note:: + + The user who owns the Nginx process, normally ``www-data`` or ``nginx``, + requires execute permission on the directories ``static``, + ``public``, ``theme_static`` and ``plugin_static`` plus all their + parent directories. This user also requires read permission on all + the files within these directories. This is normally the default. + +Nginx is now configured to serve the MediaGoblin application. Perform a quick +test to ensure that this configuration works:: + + nginx -t + +If you encounter any errors, review your Nginx configuration files, and try to +resolve them. If you do not encounter any errors, you can start your Nginx +server with one of the following commands (depending on your environment):: sudo /etc/init.d/nginx restart sudo /etc/rc.d/nginx restart + sudo systemctl restart nginx Now start MediaGoblin. Use the following command sequence as an example:: cd /srv/mediagoblin.example.org/mediagoblin/ + su mediagoblin -s /bin/bash ./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543 Visit the site you've set up in your browser by visiting @@ -426,6 +514,29 @@ Instructions and scripts for running MediaGoblin on an Apache server can be found on the `MediaGoblin wiki <http://wiki.mediagoblin.org/Deployment>`_. +Should I Keep Open Registration Enabled? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Unfortunately, in this current release of MediaGoblin we are suffering +from spammers registering to public instances en masse. As such, you +may want to either: + +a) Disable registration on your instance and just make + accounts for people you know and trust (eg via the `gmg adduser` + command). You can disable registration in your mediagoblin.ini + like so:: + + [mediagoblin] + allow_registration = false + +b) Enable a CAPTCHA plugin. But unfortunately, though some CAPTCHA + plugins exist, for various reasons we do not have any general + recommendations we can make at this point. + +We hope to have a better solution to this situation shortly. We +apologize for the inconvenience in the meanwhile. + + Security Considerations ~~~~~~~~~~~~~~~~~~~~~~~ @@ -439,3 +550,7 @@ Security Considerations and restart the server, so that it creates a new secret key. All previous sessions will be invalidated. +.. + Local variables: + fill-column: 70 + End: diff --git a/docs/source/siteadmin/foreword.rst b/docs/source/siteadmin/foreword.rst index 4c425f8d..24282f96 100644 --- a/docs/source/siteadmin/foreword.rst +++ b/docs/source/siteadmin/foreword.rst @@ -34,13 +34,14 @@ Improving the Site Administrator's Guide There are a few ways---please pick whichever method is convenient for you! -1. Write up a bug report in the bug tracker +1. Write up a bug report in the `bug tracker`_. 2. Tell someone on IRC ``#mediagoblin`` on Freenode. -3. Write an email to the devel mailing list. +3. Write an email to the `devel mailing list`_. -Information about the bugtracker, IRC and the mailing list is all on -the `join page`_. +More information about contributing is available on the `join page`_. +.. _bug tracker: https://issues.mediagoblin.org/ +.. _devel mailing list: http://lists.mediagoblin.org/listinfo/devel .. _join page: http://mediagoblin.org/join/ Patches are the most helpful, but even feedback on what you think diff --git a/docs/source/siteadmin/media-types.rst b/docs/source/siteadmin/media-types.rst index 3e8a94e9..146e1aae 100644 --- a/docs/source/siteadmin/media-types.rst +++ b/docs/source/siteadmin/media-types.rst @@ -1,6 +1,6 @@ .. MediaGoblin Documentation - Written in 2011, 2012 by MediaGoblin contributors + Written in 2011, 2012, 2014, 2015 by MediaGoblin contributors To the extent possible under law, the author(s) have dedicated all copyright and related and neighboring rights to this software to @@ -18,11 +18,11 @@ Media Types ==================== In the future, there will be all sorts of media types you can enable, -but in the meanwhile there are five additional media types: video, audio, -ascii art, STL/3d models, PDF and Document. +but in the meanwhile there are six additional media types: video, audio, +raw image, ASCII art, STL/3D models, PDF and Document. First, you should probably read ":doc:`configuration`" to make sure -you know how to modify the mediagoblin config file. +you know how to modify the MediaGoblin config file. Enabling Media Types ==================== @@ -30,7 +30,7 @@ Enabling Media Types .. note:: Media types are now plugins -Media types are enabled in your mediagoblin configuration file, typically it is +Media types are enabled in your MediaGoblin configuration file, typically it is created by copying ``mediagoblin.ini`` to ``mediagoblin_local.ini`` and then applying your changes to ``mediagoblin_local.ini``. If you don't already have a ``mediagoblin_local.ini``, create one in the way described. @@ -69,22 +69,33 @@ The file-extension-based approach is used before the sniffing-based approach, if the file-extension-based approach finds a match, the sniffing-based approach will be skipped as it uses far more processing power. +Configuring Media Types +======================= + +Each media type has a ``config_spec.ini`` file with configurable +options and comments explaining their intended side effect. For +instance the ``video`` media type configuration can be found in +``mediagoblin/media_types/video/config_spec.ini``. + Video ===== -To enable video, first install gstreamer and the python-gstreamer -bindings (as well as whatever gstremaer extensions you want, +To enable video, first install GStreamer and the python-gstreamer +bindings (as well as whatever GStreamer extensions you want, good/bad/ugly). On Debianoid systems .. code-block:: bash - sudo apt-get install python-gst0.10 \ - gstreamer0.10-plugins-base \ - gstreamer0.10-plugins-bad \ - gstreamer0.10-plugins-good \ - gstreamer0.10-plugins-ugly \ - gstreamer0.10-ffmpeg + sudo apt-get install python-gi python3-gi \ + gstreamer1.0-tools \ + gir1.2-gstreamer-1.0 \ + gir1.2-gst-plugins-base-1.0 \ + gstreamer1.0-plugins-good \ + gstreamer1.0-plugins-ugly \ + gstreamer1.0-plugins-bad \ + gstreamer1.0-libav \ + python-gst-1.0 Add ``[[mediagoblin.media_types.video]]`` under the ``[plugins]`` section in @@ -96,7 +107,7 @@ Run ./bin/gmg dbupdate -Now you should be able to submit videos, and mediagoblin should +Now you should be able to submit videos, and MediaGoblin should transcode them. .. note:: @@ -110,28 +121,18 @@ transcode them. Audio ===== -To enable audio, install the gstreamer and python-gstreamer bindings (as well -as whatever gstreamer plugins you want, good/bad/ugly), scipy and numpy are +To enable audio, install the GStreamer and python-gstreamer bindings (as well +as whatever GStreamer plugins you want, good/bad/ugly), SciPy and NumPy are also needed for the audio spectrograms. To install these on Debianoid systems, run:: - sudo apt-get install python-gst0.10 gstreamer0.10-plugins-{base,bad,good,ugly} \ - gstreamer0.10-ffmpeg python-numpy python-scipy - -The ``scikits.audiolab`` package you will install in the next step depends on the -``libsndfile1-dev`` package, so we should install it. -On Debianoid systems, run - -.. code-block:: bash - - sudo apt-get install libsndfile1-dev + sudo apt-get install python-gst-1.0 gstreamer1.0-plugins-{base,bad,good,ugly} \ + gstreamer1.0-libav python-numpy python-scipy libsndfile1-dev libasound2-dev .. note:: scikits.audiolab will display a warning every time it's imported if you do not compile it with alsa support. Alsa support is not necessary for the GNU - MediaGoblin application but if you do not wish the alsa warnings from - audiolab you should also install ``libasound2-dev`` before installing - scikits.audiolab. + MediaGoblin application. Then install ``scikits.audiolab`` for the spectrograms:: @@ -149,12 +150,34 @@ Run You should now be able to upload and listen to audio files! -Ascii art +Raw image ========= -To enable ascii art support, first install the +To enable raw image you need to install pyexiv2. On Debianoid systems + +.. code-block:: bash + + sudo apt-get install python-pyexiv2 + +Add ``[[mediagoblin.media_types.raw_image]]`` under the ``[plugins]`` +section in your ``mediagoblin_local.ini`` and restart MediaGoblin. + +Run + +.. code-block:: bash + + ./bin/gmg dbupdate + +Now you should be able to submit raw images, and MediaGoblin should +extract the JPEG preview from them. + + +ASCII art +========= + +To enable ASCII art support, first install the `chardet <http://pypi.python.org/pypi/chardet>`_ -library, which is necessary for creating thumbnails of ascii art +library, which is necessary for creating thumbnails of ASCII art .. code-block:: bash @@ -171,20 +194,20 @@ Run ./bin/gmg dbupdate -Now any .txt file you uploaded will be processed as ascii art! +Now any .txt file you uploaded will be processed as ASCII art! -STL / 3d model support +STL / 3D model support ====================== -To enable the "STL" 3d model support plugin, first make sure you have -a recentish `Blender <http://blender.org>`_ installed and available on +To enable the "STL" 3D model support plugin, first make sure you have +a recent `Blender <http://blender.org>`_ installed and available on your execution path. This feature has been tested with Blender 2.63. It may work on some earlier versions, but that is not guaranteed (and is surely not to work prior to Blender 2.5X). Add ``[[mediagoblin.media_types.stl]]`` under the ``[plugins]`` section in your -``mediagoblin_local.ini`` and restart MediaGoblin. +``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -200,9 +223,9 @@ PDF and Document To enable the "PDF and Document" support plugin, you need: -1. pdftocairo and pdfinfo for pdf only support. +1. pdftocairo and pdfinfo for PDF only support. -2. unoconv with headless support to support converting libreoffice supported +2. unoconv with headless support to support converting LibreOffice supported documents as well, such as doc/ppt/xls/odf/odg/odp and more. For the full list see mediagoblin/media_types/pdf/processing.py, unoconv_supported. @@ -215,7 +238,7 @@ To install this on Fedora: sudo yum install -y poppler-utils unoconv libreoffice-headless -Note: You can leave out unoconv and libreoffice-headless if you want only pdf +Note: You can leave out unoconv and libreoffice-headless if you want only PDF support. This will result in a much smaller list of dependencies. pdf.js relies on git submodules, so be sure you have fetched them: @@ -233,7 +256,7 @@ This feature has been tested on Fedora with: It may work on some earlier versions, but that is not guaranteed. Add ``[[mediagoblin.media_types.pdf]]`` under the ``[plugins]`` section in your -``mediagoblin_local.ini`` and restart MediaGoblin. +``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -242,3 +265,13 @@ Run ./bin/gmg dbupdate +Blog (HIGHLY EXPERIMENTAL) +========================== + +MediaGoblin has a blog media type, which you might notice by looking +through the docs! However, it is *highly experimental*. We have not +security reviewed this, and it acts in a way that is not like normal +blogs (the blog posts are themselves media types!). + +So you can play with this, but it is not necessarily recommended yet +for production use! :) diff --git a/docs/source/siteadmin/plugins.rst b/docs/source/siteadmin/plugins.rst index baca381d..8682b0c7 100644 --- a/docs/source/siteadmin/plugins.rst +++ b/docs/source/siteadmin/plugins.rst @@ -55,7 +55,7 @@ offer for your media), we would do:: virtual environment before installing with pip. Otherwise the plugin may get installed in a different environment than the one MediaGoblin is installed in. Also make sure, you use e.g. pip-2.7 if your default - python (and thus pip) is python 3 (e.g. in Ubuntu). + python (and thus pip) is python 3 (e.g. in *buntu). Once you've installed the plugin software, you need to tell MediaGoblin that this is a plugin you want MediaGoblin to use. To do @@ -110,11 +110,32 @@ comments making the bits clearer):: Check the plugin's documentation for what configuration options are available. +Once you've set up your plugin, you should be sure to update the +database to accommodate the new plugins:: -Removing plugins -================ + ./bin/gmg dbupdate -To remove a plugin, use ``pip uninstall``. For example:: + +Deactivating plugins +==================== + +You should be aware that once you enable a plugin, deactivating it +might be a bit tricky, for migrations reasons. In the future we may +produce better tooling to accommodate this. In short, you will need to +do a bit of database surgery by: + +- Removing all tables and indexes installed by the plugin +- Removing the plugin's migration head id from the `alembic_version` + table. (You might be able to determine which to remove via + examining the output of `./bin/gmg alembic heads`) + +Note that this is a VERY TRICKY process, and you should be sure to make +a backup first. You've been warned! + +Removing plugin packages +======================== + +To remove an external plugin's package, use ``pip uninstall``. For example:: pip uninstall mediagoblin-licenses diff --git a/docs/source/siteadmin/production-deployments.rst b/docs/source/siteadmin/production-deployments.rst index 839d3ce5..ee915573 100644 --- a/docs/source/siteadmin/production-deployments.rst +++ b/docs/source/siteadmin/production-deployments.rst @@ -1,6 +1,6 @@ .. MediaGoblin Documentation - Written in 2011, 2012 by MediaGoblin contributors + Written in 2011, 2012, 2013, 2014, 2015 by MediaGoblin contributors To the extent possible under law, the author(s) have dedicated all copyright and related and neighboring rights to this software to @@ -19,15 +19,145 @@ This document contains a number of suggestions for deploying MediaGoblin in actual production environments. Consider ":doc:`deploying`" for a basic overview of how to deploy MediaGoblin. -Deploy with Paste +Deploy with paste ----------------- The MediaGoblin WSGI application instance you get with ``./lazyserver.sh`` is not ideal for a production MediaGoblin deployment. Ideally, you should be able -to use an "init" or "control" script to launch and restart the MediaGoblin -process. +to use a Systemd service file or an init script to launch and restart the +MediaGoblin process. -Use the following command as the basis for such a script: +We will explore setting up MediaGoblin Systemd service files and init scripts, +but first we need to create the directory that will store the MediaGoblin logs. + + +.. _create-log-file-dir: + +Create the directory for your log file: +--------------------------------------- + +Production logs for the MediaGoblin application are kept in the +``/var/log/mediagoblin`` directory. Create the directory and give it the +proper permissions:: + + sudo mkdir -p /var/log/mediagoblin && sudo chown -hR mediagoblin:mediagoblin /var/log/mediagoblin + + +.. _systemd-service-files: + +Use Systemd service files +------------------------- + +If your operating system uses Systemd, you can use Systemd ``service files`` +to manage both the Celery and Paste processes. Place the following service +files in the ``/etc/systemd/system/`` directory. + +The first file should be named ``mediagoblin-celeryd.service``. Be sure to +modify it to suit your environment's setup: + +.. code-block:: bash + + # Set the WorkingDirectory, Environment and ExecStart values to match your environment. + # If using Debian/*buntu, mkdir and chown are located in /bin/mkdir and /bin/chown, respectively. + # If using Fedora/CentOS/Red Hat, mkdir and chown are located in /usr/bin/mkdir and /usr/bin/chown, respectively. + + [Unit] + Description=MediaGoblin Celeryd + + [Service] + User=mediagoblin + Group=mediagoblin + Type=simple + WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin + # Start mg-celeryd process as root, then switch to mediagoblin user/group + # (This is needed to run the ExecStartPre commands) + PermissionsStartOnly=true + # Create directory for PID (if needed) and set ownership + ExecStartPre=/bin/mkdir -p /run/mediagoblin + ExecStartPre=/bin/chown -hR mediagoblin:mediagoblin /run/mediagoblin + # Celery process will run as the `mediagoblin` user after start. + Environment=MEDIAGOBLIN_CONFIG=/srv/mediagoblin.example.org/mediagoblin/mediagoblin_local.ini \ + CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery + ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/celery worker \ + --logfile=/var/log/mediagoblin/celery.log \ + --loglevel=INFO + PIDFile=/run/mediagoblin/mediagoblin-celeryd.pid + + [Install] + WantedBy=multi-user.target + + +The second file should be named ``mediagoblin-paster.service``: + + +.. code-block:: bash + + # Set the WorkingDirectory, Environment and ExecStart values to match your environment. + # If using Debian/*buntu, mkdir and chown are located in /bin/mkdir and /bin/chown, respectively. + # If using Fedora/CentOS/Red Hat, mkdir and chown are located in /usr/bin/mkdir and /usr/bin/chown, respectively. + [Unit] + Description=Mediagoblin + + [Service] + Type=forking + User=mediagoblin + Group=mediagoblin + Environment=CELERY_ALWAYS_EAGER=false + WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin + # Start mg-paster process as root, then switch to mediagoblin user/group + PermissionsStartOnly=true + ExecStartPre=-/bin/mkdir -p /run/mediagoblin + ExecStartPre=/bin/chown -hR mediagoblin:mediagoblin /run/mediagoblin + + ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ + /srv/mediagoblin.example.org/mediagoblin/paste_local.ini \ + --pid-file=/var/run/mediagoblin/mediagoblin.pid \ + --log-file=/var/log/mediagoblin/mediagoblin.log \ + --daemon \ + --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543 + ExecStop=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ + --pid-file=/var/run/mediagoblin/mediagoblin.pid \ + /srv/mediagoblin.example.org/mediagoblin/paste_local.ini stop + PIDFile=/var/run/mediagoblin/mediagoblin.pid + + [Install] + WantedBy=multi-user.target + + + +Enable these processes to start at boot by entering:: + + sudo systemctl enable mediagoblin-celeryd.service && sudo systemctl enable mediagoblin-paster.service + + +Start the processes for the current session with:: + + sudo systemctl start mediagoblin-paster.service + sudo systemctl start mediagoblin-celeryd.service + + +If either command above gives you an error, you can investigate the cause of +the error by entering:: + + sudo systemctl status mediagoblin-celeryd.service or + sudo systemctl status mediagoblin-paster.service + +The above ``systemctl status`` command is also useful if you ever want to +confirm that a process is still running. If you make any changes to the service +files, you can reload the service files by entering:: + + sudo systemctl daemon-reload + +After entering that command, you can attempt to start the Celery or Paste +processes again. + +.. _init-script: + +Use an init script +------------------ + +If your system does not use Systemd, you can use the following command as the +basis for an init script: .. code-block:: bash @@ -53,7 +183,30 @@ as the basis for your script: --pid-file=/var/run/mediagoblin.pid \ --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543 -Separate Celery + +Members of the MediaGoblin community have provided init scripts for the +following GNU/Linux distributions: + +Debian + * `GNU MediaGoblin init scripts + <https://github.com/joar/mediagoblin-init-scripts>`_ + by `Joar Wandborg <http://wandborg.se>`_ + +Arch Linux + * `MediaGoblin - ArchLinux rc.d scripts + <http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts>`_ + by `Jeremy Pope <http://jpope.org/>`_ + * `Mediagoblin init script on Archlinux + <http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/>`_ + by `Chimo <http://chimo.chromic.org/>`_ + +You can reference these scripts to create an init script for your own operating +system. Similar scripts will be in your system's ``/etc/init.d/`` +or ``/etc/rc.d/`` directory, but the specifics of an init script will vary from +one distribution to the next. + + +Separate celery --------------- MediaGoblin uses `Celery`_ to handle heavy and long-running tasks. Celery can @@ -91,6 +244,11 @@ To launch Celery separately from the MediaGoblin WSGI application: CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd +If you use our example Systemd ``service files``, Celery will be set to the +"CELERY_ALWAYS_EAGER=false" value by default. This will provide your users +with the best user experience, as all media processing will be done in the +background. + .. _sentry: Set up sentry to monitor exceptions @@ -102,31 +260,6 @@ documentation. .. _`raven`: http://raven.readthedocs.org -.. _init-script: - -Use an Init Script ------------------- - -Look in your system's ``/etc/init.d/`` or ``/etc/rc.d/`` directory for -examples of how to build scripts that will start, stop, and restart -MediaGoblin and Celery. These scripts will vary by -distribution/operating system. - -These are scripts provided by the MediaGoblin community: - -Debian - * `GNU MediaGoblin init scripts - <https://github.com/joar/mediagoblin-init-scripts>`_ - by `Joar Wandborg <http://wandborg.se>`_ - -Arch Linux - * `MediaGoblin - ArchLinux rc.d scripts - <http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts>`_ - by `Jeremy Pope <http://jpope.org/>`_ - * `Mediagoblin init script on Archlinux - <http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/>`_ - by `Chimo <http://chimo.chromic.org/>`_ - .. TODO insert init script here .. TODO are additional concerns ? .. Other Concerns diff --git a/docs/source/siteadmin/relnotes.rst b/docs/source/siteadmin/relnotes.rst index 3542bdcb..f32ca792 100644 --- a/docs/source/siteadmin/relnotes.rst +++ b/docs/source/siteadmin/relnotes.rst @@ -21,6 +21,304 @@ This chapter has important information for releases in it. If you're upgrading from a previous release, please read it carefully, or at least skim over it. +.. note:: + + ALWAYS do backups before upgrading, especially before + running migrations! That way if something goes wrong, we can fix + things! + + And be sure to shut down your current MediaGoblin/Celery processes + before upgrading! + +.. note:: + + The MediaGoblin repository used to be on gitorious.org, but since + gitorious.org shut down, we had to move. We are presently on + Savannah. You may need to update your git repository location:: + + git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git + + +0.9.0 +===== + +This release has a number of improvements, but is also a major +"plumbing upgrade" release to MediaGoblin. Notably, we now support +Python 3, which is pretty cool! + +**Do this to upgrade** + +0. If you haven't already, switch the git remote URL: + ``git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git`` +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.9.0`` +2. Run + ``./bootstrap.sh && ./configure && make`` +3. Also run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +**Bugfixes/improvements:** + +- Python 3 is now a first class citizen! We now support both + Python 2.7 and Python 3.4 or later. +- Major updates to internal tooling to pave the way for federation. + - Massive overhaul to the database layout (particularly in + permitting generic relations) + - OAuth updates + - Updating how we handle collections + - Add a "graveyard" system with tombstones for keeping information + about removed objects + - Large overhaul to how "comments" work. In federation, many things + can reply to many things, so we had to loosen the model. +- If your user has some collections available, these will be presented + as a dropdown option while submitting media. +- Begin using Alembic for migrations +- Lots of bugfixes and etc + - Many fixes to typos + - Some fixes to the blog system + - Switch to waitress for development + - And more...! + + +0.8.1 +===== + +This release is a security and bugfix release. We recommend you upgrade as +soon as possible. + +**Do this to upgrade** + +0. If you haven't already, switch the git remote URL: + ``git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git`` +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.8.1`` +2. Run + ``./bootstrap.sh && ./configure && make`` +3. Also run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +(Please check intermediate release steps as well if not upgrading from +0.8.0) + +**Bugfixes/improvements:** + +Most importantly, there is an **important security fix**: + +Quoting here a portion of the +`release blogpost <http://mediagoblin.org/news/mediagoblin-0.8.1-security-release.html>`_:: + + We have had a security problem in our OAuth implementation reported to + us privately and have taken steps to address it. The security problem + affects all versions of GNU MediaGoblin since 0.5.0. I have created a patch + for this and released a minor version 0.8.1. It's strongly advised + that everyone upgrade as soon as they can. + + In order to exploit the security issue, an attacker must have had + access to a logged in session to your GNU MediaGoblin account. If you + have kept your username and password secret, logging in only over + HTTPS and you've not left yourself logged in on publicly accessible + computers, you should be safe. However it's still advised all users + take the following precautions, listed below. + + Users should check their authorized clients. Any client which looks + unfamiliar to you, you should deauthorize. To check this: + + 1) Log in to the GNU MediaGoblin instance + 2) Click the drop down arrow in the upper right + 3) Click "Change account settings" + 4) At the bottom click the "Deauthorize applications" link + + If you are unsure of any of these, click "Deauthorize". + +There are other bugfixes, but they are fairly minor. + + +0.8.0 +===== + +This release has a number of changes related to the way we recommend +building MediaGoblin; upgrade steps are below, but if you run into +trouble, consider pinging the MediaGoblin list or IRC channel. + +**Do this to upgrade** + +0. If you haven't already, switch the git remote URL: + ``git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git`` +1. If you don't have node.js installed, you'll need it for handling + MediaGoblin's static web dependencies. Install this via your + distribution! (In the glorious future MediaGoblin will be simply + packaged for your distribution so you won't have to worry about + this!) +2. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.8.0`` +3. Run + ``./bootstrap.sh && ./configure && make`` +4. Also run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +Please note the important steps of 0 and 2, which have not appeared in +prior upgrade guides! + +Additionally: + +- Are you using audio or video media types? In that case, you'll need + to update your GStreamer instance to 1.0. +- The Pump API needs some data passed through to the WSGI application, + so if you are using Apache with mod_wsgi you should be sure to make + sure to add "WSGIPassAuthorization On" to your config. (Using the + default MediaGoblin documentation and config, things should work + as-is.) + + +**Bugfixes/improvements:** + +- Preliminary / experimental support for Python 3! +- Footer forced to the bottom of page +- Massive improvements to Pump API support + - Able to run on multiple existing Pump clients! Including Pumpa + and Dianara! +- much cleaner ./configure && make support; it's now the default +- Clearer documentation on permissions and installation +- Switched from Transifex, which had become proprietary, to an + instance of Pootle hosted for GNU +- Moved to GStreamer 1.0! This also adds a new thumbnailer which + gives much better results in +- Removed terrible check-JavaScript-dependencies-into-your-application + setup, now using Bower for dependency tracking +- Put some scaffolding in place for Alembic, which will be used for + future migration work +- Automatically create a fresh mediagoblin.ini from + mediagoblin.ini.example +- no more need for mediagoblin_local.ini (though it's still supported) +- Fix lowercasing of username in auth steps +- Slowly moving towards removing global state (a source of many bugs) + +0.7.1 +===== + +This is a purely bugfix release. Important changes happened since +0.7.0; if running MediaGoblin 0.7.0, an upgrade is highly recommended; +see below. This release is especially useful if you have been running +PostgreSQL and have been receiving seemingly random database transaction +errors. + +**Do this to upgrade** + +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.7.1 && git submodule init && git submodule update`` +2. Make sure to run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +That's it, probably! If you run into problems, don't hesitate to +`contact us <http://mediagoblin.org/pages/join.html>`_ +(IRC is often best). + +**Bugfixes/improvements:** + +- The *MOST IMPORTANT* change in this release: + Disabling a couple of non-critical features that were causing + database transaction issues. (These should be back by 0.8.0.) + + + Disabled the "checking if the database is up to date at + MediaGoblin startup" feature + + Disabled the garbage collection stuff by default for now + (You can set garbage_collection under the config MediaGoblin + header to something other than 0 to turn it back on for now, but + it's potentially risky for the moment.) + +- Some fixes to the 0.7.0 docs +- Fixed Sandy 70s speedboat navbar by updating git submodule +- Added support for cr2 files in raw_image media type +- Added a description to setup.py +- Collection and CollectionItem objects now have nicer in-python representations +- Fixed Unicode error with raw image mediatype logging +- Fixed #945 "Host metadata does not confirm to spec (/.well-known/meta-data)" + + + Add XRD+XML formatting for /.well-known/host-meta + + Add /.well-known/webfinger API to lookup user hrefs + +- deleteuser gmg subcommand now fails gracefully +- Removed a false download link from setup.py + +0.7.0 +===== + +**Do this to upgrade** + +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.7.0 && git submodule init && git submodule update`` +2. Make sure to run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +(NOTE: earlier versions of the 0.7.0 release instructions left out the +``git submodule init`` step! If you did an upgrade earlier based on +these instructions and your theme looks weirdly aligned, try running +the following:) + + ``git submodule init && git submodule update`` + +That's it, probably! If you run into problems, don't hesitate to +`contact us <http://mediagoblin.org/pages/join.html>`_ +(IRC is often best). + +**New features:** + +- New mobile upload API making use of the + `Pump API <https://github.com/e14n/pump.io/blob/master/API.md>`_ + (which will be the foundation for MediaGoblin's federation) +- New theme: Sandy 70s Speedboat! + +- Metadata features! We also now have a JSON-LD context. + +- Many improvements for archival institutions, including metadata + support and featuring items on the homepage. With the (new!) + archivalook plugin enabled, featuring media is possible. + Additionally, metadata about the particular media item will show up + in the sidebar. + + In the future these plugins may be separated, but for now they have + come together as part of the same plugin. + +- There is a new gmg subcommand called batchaddmedia that allows for + uploading many files at once. This is aimed to be useful for + archival institutions and groups where there is an already existing + and large set of available media that needs to be included. +- Speaking of, the call to PostgreSQL in the makefile is fixed. +- We have a new, generic media-page context hook that allows for + adding context depending on the type of media. +- Tired of video thumbnails breaking during processing all the time? + Good news, everyone! Video thumbnail generation should not fail + frequently anymore. (We think...) +- You can now set default permissions for new users in the config. + +- bootstrap.sh / gnu configuration stuff still exists, but moves to be + experimental-bootstrap.sh so as to not confuse newcomers. There are + some problems currently with the autoconf stuff that we need to work + out... we still have interest in supporting it, though help is + welcome. + +- MediaGoblin now checks whether or not the database is up to date + when starting. +- Switched to `Skeleton <http://www.getskeleton.com/>`_ as a system for + graphic design. +- New gmg subcommands for administrators: + - A "deletemedia" command + - A "deleteuser" command +- We now have a blogging media type... it's very experimental, + however. Use with caution! +- We have switched to exifread as an external library for reading EXIF + data. It's basically the same thing as before, but packaged + separately from MediaGoblin. +- Many improvements to internationalization. Also (still rudimentary, + but existent!) RTL language support! + +**Known issues:** + - The host-meta is now JSON by default; in the spec it should be XML by + default. We have done this because of compatibility with the pump + API. We are checking with upstream to see if there is a way to + resolve this discrepancy. + + 0.6.1 ===== @@ -62,7 +360,7 @@ nickname "Lore of the Admin"! - New tools to control how much users can upload, both as a general user limit, or per file. - You can set this with the following options in your mediagoblin + You can set this with the following options in your MediaGoblin config file: `upload_limit` and `max_file_size`. Both are integers in megabytes. @@ -70,7 +368,7 @@ nickname "Lore of the Admin"! upload too, though an interface for this is not yet exposed. See the "uploaded" field on the core__users table. -- MediaGoblin now contains an authentication plugin for ldap! You +- MediaGoblin now contains an authentication plugin for LDAP! You can turn on the mediagoblin.plugins.ldap plugin to make use of this. See the documentation: :ref:`ldap-plugin` @@ -125,8 +423,8 @@ v0.5.1 is a bugfix release... the steps are the same as for 0.5.1. ===== **NOTE:** If using the API is important to you, we're in a state of -ransition towards a new API via the Pump API. As such, though the old -API still probably works, some changes have happened to the way oauth +transition towards a new API via the Pump API. As such, though the old +API still probably works, some changes have happened to the way OAuth works to make it more Pump-compatible. If you're heavily using clients using the old API, you may wish to hold off on upgrading for now. Otherwise, jump in and have fun! :) @@ -171,21 +469,21 @@ If you run into problems, don't hesitate to * Comment preview! * Users now have the ability to change their email associated with their account. -* New oauth code as we move closer to federation support. -* Experimental pyconfigure support for GNU-style configue and makefile +* New OAuth code as we move closer to federation support. +* Experimental pyconfigure support for GNU-style configure and makefile deployment. * Database foundations! You can now pre-populate the database models. * Way faster unit test run-time via in-memory database. * All mongokit stuff has been cleaned up. -* Fixes for non-ascii filenames. +* Fixes for non-ASCII filenames. * The option to stay logged in. -* Mediagoblin has been upgraded to use the latest `celery <http://celeryproject.org/>`_ +* MediaGoblin has been upgraded to use the latest `Celery <http://celeryproject.org/>`_ version. * You can now add jinja2 extensions to your config file to use in custom templates. * Fixed video permission issues. -* Mediagoblin docs are now hosted with multiple versions. -* We removed redundent tooltips from the STL media display. +* MediaGoblin docs are now hosted with multiple versions. +* We removed redundant tooltips from the STL media display. * We are now using itsdangerous for verification tokens. @@ -197,7 +495,7 @@ fix in the newly released document support which prevented the "conversion via libreoffice" feature. If you were running 0.4.0 you can upgrade to v0.4.1 via a simple -switch and restarting mediagoblin/celery with no other actions. +switch and restarting MediaGoblin/Celery with no other actions. Otherwise, follow 0.4.0 instructions. @@ -216,7 +514,7 @@ Otherwise, follow 0.4.0 instructions. Keep on reading to hear more about new plugin features. 4. If you want to take advantage of new plugins that have statically served assets, you are going to need to add the new "plugin_static" - section to your nginx config. Basically the following for nginx:: + section to your Nginx config. Basically the following for Nginx:: # Plugin static files (usually symlinked in) location /plugin_static/ { @@ -259,7 +557,7 @@ please note the following: date of an image when available (available as the "original_date_visible" variable) * Moved unit testing system from nosetests to py.test so we can better - handle issues with sqlalchemy exploding with different database + handle issues with SQLAlchemy exploding with different database configurations. Long story :) * You can now disable the ability to post comments. * Tags now can be up to length 255 characters by default. @@ -289,7 +587,7 @@ you run into problems, don't hesitate to * New dropdown menu for accessing various features. -* Significantly improved URL generation. Now mediagoblin won't give +* Significantly improved URL generation. Now MediaGoblin won't give up on making a slug if it looks like there will be a duplicate; it'll try extra hard to generate a meaningful one instead. @@ -297,13 +595,13 @@ you run into problems, don't hesitate to linking to a slug; /u/username/m/id:35/ is the kind of reference we now use to linking to entries with ids. However, old links with entries that linked to ids should work just fine with our migration. - The only urls that might break in this release are ones using colons + The only URLs that might break in this release are ones using colons or equal signs. * New template hooks for plugin authoring. * As a demonstration of new template hooks for plugin authoring, - openstreetmap support now moved to a plugin! + OpenStreetMap support now moved to a plugin! * Method to add media to collections switched from icon of paperclip to button with "add to collection" text. @@ -314,9 +612,9 @@ you run into problems, don't hesitate to waste gobs of memory. * Video transcoding now optional for videos that meet certain - criteria. By default, MediaGoblin will not transcode webm videos + criteria. By default, MediaGoblin will not transcode WebM videos that are smaller in resolution than the MediaGoblin defaults, and - MediaGoblin can also be configured to allow theora files to not be + MediaGoblin can also be configured to allow Theora files to not be transcoded as well. * Per-user license preference option; always want your uploads to be @@ -346,7 +644,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. **Do this to upgrade** - # directory of your mediagoblin install + # directory of your MediaGoblin install cd /srv/mediagoblin.example.org # copy source for this release @@ -366,13 +664,13 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. * **3d model support!** You can now upload STL and OBJ files and display them in - MediaGoblin. Requires a recent-ish Blender; for details see: + MediaGoblin. Requires a recent Blender; for details see: :ref:`deploying-chapter` * **trim_whitespace** We bundle the optional plugin trim_whitespace which reduces the size - of the delivered html output by reducing redundant whitespace. + of the delivered HTML output by reducing redundant whitespace. See :ref:`core-plugin-section` for plugin documentation @@ -386,7 +684,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. and `OMGMG <https://github.com/jwandborg/omgmg>`_, an example of a web application hooking up to the API. - This is a plugin, so you have to enable it in your mediagoblin + This is a plugin, so you have to enable it in your MediaGoblin config file by adding a section under [plugins] like:: [plugins] @@ -399,7 +697,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. For applications that use OAuth to connect to the API. - This is a plugin, so you have to enable it in your mediagoblin + This is a plugin, so you have to enable it in your MediaGoblin config file by adding a section under [plugins] like:: [plugins] @@ -419,7 +717,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. Geolocation is also now on by default. -* **Miscelaneous visual improvements** +* **Miscellaneous visual improvements** We've made a number of small visual improvements including newer and nicer looking thumbnails and improved checkbox placement. @@ -434,7 +732,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. 1. Make sure to run ``bin/gmg dbuptdate`` after upgrading. 2. If you set up your server config with an older version of - mediagoblin and the mediagoblin docs, it's possible you don't + MediaGoblin and the MediaGoblin docs, it's possible you don't have the "theme static files" alias, so double check to make sure that section is there if you are having problems. diff --git a/docs/source/siteadmin/theming.rst b/docs/source/siteadmin/theming.rst index 11ae3875..9c01a5b3 100644 --- a/docs/source/siteadmin/theming.rst +++ b/docs/source/siteadmin/theming.rst @@ -73,9 +73,9 @@ want to install this theme! Don't worry, it's fairly painless. Set up your webserver to serve theme assets ------------------------------------------- -If you followed the nginx setup example in :ref:`webserver-config` you +If you followed the Nginx setup example in :ref:`webserver-config` you should already have theme asset setup. However, if you set up your -server config with an older version of mediagoblin and the mediagoblin +server config with an older version of MediaGoblin and the MediaGoblin docs, it's possible you don't have the "theme static files" alias, so double check to make sure that section is there if you are having problems. @@ -103,7 +103,7 @@ Other variables you may consider setting: `theme_web_path` When theme-specific assets are specified, this is where MediaGoblin - will set the urls. By default this is ``"/theme_static/"`` so in + will set the URLs. By default this is ``"/theme_static/"`` so in the case that your theme was trying to access its file ``"images/shiny_button.png"`` MediaGoblin would link to ``/theme_static/images/shiny_button.png``. @@ -136,7 +136,7 @@ if necessary):: | | |- im_a_hedgehog.png # hedgehog-containing image used by theme | | '- custom_logo.png # your theme's custom logo | '- css/ - | '- hedgehog.css # your site's hedgehog-specific css + | '- hedgehog.css # your site's hedgehog-specific CSS |- README.txt # Optionally, a readme file (not required) |- AGPLv3.txt # AGPL license file for your theme. (good practice) '- CC0_1.0.txt # CC0 1.0 legalcode for the assets [if appropriate!] @@ -164,7 +164,7 @@ Only a few things need to go in here:: [theme] name = Hedgehog-ification description = For hedgehog lovers ONLY - licensing = AGPLv3 or later templates; assets (images/css) waived under CC0 1.0 + licensing = AGPLv3 or later templates; assets (images/CSS) waived under CC0 1.0 The name and description fields here are to give users an idea of what your theme is about. For the moment, we don't have any listing @@ -232,7 +232,7 @@ You should include AGPLv3.txt with your theme as this is required for the assets. You can copy this from ``mediagoblin/licenses/``. In the above example, we also use CC0 to waive our copyrights to -images and css, so we also included CC0_1.0.txt +images and CSS, so we also included CC0_1.0.txt A README.txt file @@ -247,7 +247,7 @@ Simple theming by adding CSS ---------------------------- Many themes won't require anything other than the ability to override -some of MediaGoblin's core css. Thankfully, doing so is easy if you +some of MediaGoblin's core CSS. Thankfully, doing so is easy if you combine the above steps! In your theme, do the following (make sure you make the necessary @@ -260,7 +260,7 @@ Great, now open that file and add something like this at the end:: <link rel="stylesheet" type="text/css" href="{{ request.staticdirect('/css/theme.css', 'theme') }}"/> -You can name the css file whatever you like. Now make the directory +You can name the CSS file whatever you like. Now make the directory for ``assets/css/`` and add the file ``assets/css/theme.css``. You can now put custom CSS files in here and any CSS you add will |