diff options
Diffstat (limited to 'docs/source/siteadmin')
| -rw-r--r-- | docs/source/siteadmin/codebase.rst | 160 | ||||
| -rw-r--r-- | docs/source/siteadmin/deploying.rst | 70 | ||||
| -rw-r--r-- | docs/source/siteadmin/media-types.rst | 112 | ||||
| -rw-r--r-- | docs/source/siteadmin/plugins.rst | 22 | ||||
| -rw-r--r-- | docs/source/siteadmin/production-deployments.rst | 13 | ||||
| -rw-r--r-- | docs/source/siteadmin/relnotes.rst | 160 |
6 files changed, 355 insertions, 182 deletions
diff --git a/docs/source/siteadmin/codebase.rst b/docs/source/siteadmin/codebase.rst deleted file mode 100644 index 3ef91290..00000000 --- a/docs/source/siteadmin/codebase.rst +++ /dev/null @@ -1,160 +0,0 @@ -.. MediaGoblin Documentation - - Written in 2011, 2012 by MediaGoblin contributors - - To the extent possible under law, the author(s) have dedicated all - copyright and related and neighboring rights to this software to - the public domain worldwide. This software is distributed without - any warranty. - - You should have received a copy of the CC0 Public Domain - Dedication along with this software. If not, see - <http://creativecommons.org/publicdomain/zero/1.0/>. - -.. _codebase-chapter: - -======================== - Codebase Documentation -======================== - -.. contents:: Sections - :local: - - -This chapter covers the libraries that GNU MediaGoblin uses as well as -various recipes for getting things done. - -.. Note:: - - This chapter is in flux. Clearly there are things here that aren't - documented. If there's something you have questions about, please - ask! - - See `the join page on the website <http://mediagoblin.org/join/>`_ - for where we hang out. - -For more information on how to get started hacking on GNU MediaGoblin, -see `the wiki <http://wiki.mediagoblin.org/>`_. - - -Software Stack -============== - -* Project infrastructure - - * `Python <http://python.org/>`_: the language we're using to write - this - - * `Nose <http://somethingaboutorange.com/mrl/projects/nose/>`_: - for unit tests - - * `virtualenv <http://www.virtualenv.org/>`_: for setting up an - isolated environment to keep mediagoblin and related packages - (potentially not required if MediaGoblin is packaged for your - distro) - -* Data storage - - * `SQLAlchemy <http://sqlalchemy.org/>`_: SQL ORM and database - interaction library for Python. Currently we support sqlite and - postgress as backends. - -* Web application - - * `Paste Deploy <http://pythonpaste.org/deploy/>`_ and - `Paste Script <http://pythonpaste.org/script/>`_: we'll use this for - configuring and launching the application - - * `WebOb <http://pythonpaste.org/webob/>`_: nice abstraction layer - from HTTP requests, responses and WSGI bits - - * `Routes <http://routes.groovie.org/>`_: for URL routing - - * `Beaker <http://beaker.groovie.org/>`_: for handling sessions and - caching - - * `Jinja2 <http://jinja.pocoo.org/docs/>`_: the templating engine - - * `WTForms <http://wtforms.simplecodes.com/>`_: for handling, - validation, and abstraction from HTML forms - - * `Celery <http://celeryproject.org/>`_: for task queuing (resizing - images, encoding video, ...) - - * `Babel <http://babel.edgewall.org>`_: Used to extract and compile - translations. - - * `Markdown (for python) <http://pypi.python.org/pypi/Markdown>`_: - implementation of `Markdown <http://daringfireball.net/projects/markdown/>`_ - text-to-html tool to make it easy for people to write richtext - comments, descriptions, and etc. - - * `lxml <http://lxml.de/>`_: nice xml and html processing for - python. - -* Media processing libraries - - * `Python Imaging Library <http://www.pythonware.com/products/pil/>`_: - used to resize and otherwise convert images for display. - - * `GStreamer <http://gstreamer.freedesktop.org/>`_: (Optional, for - video hosting sites only) Used to transcode video, and in the - future, probably audio too. - - * `chardet <http://pypi.python.org/pypi/chardet>`_: (Optional, for - ascii art hosting sites only) Used to make ascii art thumbnails. - -* Front end - - * `JQuery <http://jquery.com/>`_: for groovy JavaScript things - - - -What's where -============ - -After you've run checked out mediagoblin and followed the virtualenv -instantiation instructions, you're faced with the following directory -tree:: - - mediagoblin/ - |- mediagoblin/ # source code - | |- tests/ - | |- templates/ - | |- auth/ - | \- submit/ - |- docs/ # documentation - |- devtools/ # some scripts for developer convenience - | - | # the below directories are installed into your virtualenv checkout - | - |- bin/ # scripts - |- develop-eggs/ - |- lib/ # python libraries installed into your virtualenv - |- include/ - |- mediagoblin.egg-info/ - |- parts/ - |- user_dev/ # sessions, etc - - -As you can see, all the code for GNU MediaGoblin is in the -``mediagoblin`` directory. - -Here are some interesting files and what they do: - -:routing.py: maps url paths to views -:views.py: views handle http requests -:models.py: holds the sqlalchemy schemas---these are the data structures - we're working with - -You'll notice that there are several sub-directories: tests, -templates, auth, submit, ... - -``tests`` holds the unit test code. - -``templates`` holds all the templates for the output. - -``auth`` and ``submit`` are modules that enacpsulate authentication -and media item submission. If you look in these directories, you'll -see they have their own ``routing.py``, ``view.py``, and -``models.py`` in addition to some other code. diff --git a/docs/source/siteadmin/deploying.rst b/docs/source/siteadmin/deploying.rst index a270c723..9bcc0637 100644 --- a/docs/source/siteadmin/deploying.rst +++ b/docs/source/siteadmin/deploying.rst @@ -32,6 +32,11 @@ GNU/Linux distro. install. If instead you want to join in as a contributor, see our `Hacking HOWTO <http://wiki.mediagoblin.org/HackingHowto>`_ instead. + There are also many ways to install servers... for the sake of + simplicity, our instructions below describe installing with nginx. + For more recipes, including Apache, see + `our wiki <http://wiki.mediagoblin.org/Deployment>`_. + Prepare System -------------- @@ -165,7 +170,7 @@ And set up the in-package virtualenv:: If you have problems here, consider trying to install virtualenv with the ``--distribute`` or ``--no-site-packages`` options. If - your system's default Python is in the 3.x series you man need to + your system's default Python is in the 3.x series you may need to run ``virtualenv`` with the ``--python=python2.7`` or ``--python=python2.6`` options. @@ -173,22 +178,50 @@ The above provides an in-package install of ``virtualenv``. While this is counter to the conventional ``virtualenv`` configuration, it is more reliable and considerably easier to configure and illustrate. If you're familiar with Python packaging you may consider deploying with -your preferred the method. +your preferred method. Assuming you are going to deploy with FastCGI, you should also install flup:: ./bin/easy_install flup +(Sometimes this breaks because flup's site is flakey. If it does for +you, try):: + + ./bin/easy_install https://pypi.python.org/pypi/flup/1.0.3.dev-20110405 + This concludes the initial configuration of the development environment. In the future, when you update your codebase, you should also run:: ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate +Note: If you are running an active site, depending on your server +configuration, you may need to stop it first or the dbupdate command +may hang (and it's certainly a good idea to restart it after the +update) + + Deploy MediaGoblin Services --------------------------- +Edit site configuration +~~~~~~~~~~~~~~~~~~~~~~~ + +A few basic properties must be set before MediaGoblin will work. First +make a copy of ``mediagoblin.ini`` for editing so the original config +file isn't lost:: + + cp mediagoblin.ini mediagoblin_local.ini + +Then: + - Set ``email_sender_address`` to the address you wish to be used as + the sender for system-generated emails + - Edit ``direct_remote_path``, ``base_dir``, and ``base_url`` if + your mediagoblin directory is not the root directory of your + vhost. + + Configure MediaGoblin to use the PostgreSQL database ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -224,11 +257,11 @@ browser to confirm that the service is operable. .. _webserver-config: -Connect the Webserver to MediaGoblin with FastCGI -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This section describes how to configure MediaGoblin to work via -FastCGI. Our configuration example will use nginx, however, you may +FastCGI and nginx +~~~~~~~~~~~~~~~~~ + +This configuration example will use nginx, however, you may use any webserver of your choice as long as it supports the FastCGI protocol. If you do not already have a web server, consider nginx, as the configuration files may be more clear than the @@ -271,6 +304,10 @@ this ``nginx.conf`` file should be modeled on the following:: # Change this to update the upload size limit for your users client_max_body_size 8m; + # prevent attacks (someone uploading a .txt file that the browser + # interprets as an HTML file, etc.) + add_header X-Content-Type-Options nosniff; + server_name mediagoblin.example.org www.mediagoblin.example.org; access_log /var/log/nginx/mediagoblin.example.access.log; error_log /var/log/nginx/mediagoblin.example.error.log; @@ -325,3 +362,24 @@ Visit the site you've set up in your browser by visiting smaller deployments. However, for larger production deployments with larger processing requirements, see the ":doc:`production-deployments`" documentation. + + +Apache +~~~~~~ + +Instructions and scripts for running MediaGoblin on an Apache server +can be found on the `MediaGoblin wiki <http://wiki.mediagoblin.org/Deployment>`_. + + +Security Considerations +~~~~~~~~~~~~~~~~~~~~~~~ + +.. warning:: + + The directory ``user_dev/crypto/`` contains some very + sensitive files. + Especially the ``itsdangeroussecret.bin`` is very important + for session security. Make sure not to leak its contents anywhere. + If the contents gets leaked nevertheless, delete your file + and restart the server, so that it creates a new secret key. + All previous sessions will be invalifated then. diff --git a/docs/source/siteadmin/media-types.rst b/docs/source/siteadmin/media-types.rst index 5653217f..210094b9 100644 --- a/docs/source/siteadmin/media-types.rst +++ b/docs/source/siteadmin/media-types.rst @@ -43,6 +43,15 @@ video media types, then the list would look like this:: media_types = mediagoblin.media_types.image, mediagoblin.media_types.video +Note that after enabling new media types, you must run dbupdate like so:: + + ./bin/gmg dbupdate + +If you are running an active site, depending on your server +configuration, you may need to stop it first (and it's certainly a +good idea to restart it after the update). + + How does MediaGoblin decide which media type to use for a file? =============================================================== @@ -62,12 +71,27 @@ Video To enable video, first install gstreamer and the python-gstreamer bindings (as well as whatever gstremaer extensions you want, -good/bad/ugly). On Debianoid systems:: +good/bad/ugly). On Debianoid systems - sudo apt-get install python-gst0.10 gstreamer0.10-plugins-{base,bad,good,ugly} \ +.. code-block:: bash + + sudo apt-get install python-gst0.10 \ + gstreamer0.10-plugins-base \ + gstreamer0.10-plugins-bad \ + gstreamer0.10-plugins-good \ + gstreamer0.10-plugins-ugly \ gstreamer0.10-ffmpeg +Add ``mediagoblin.media_types.video`` to the ``media_types`` list in your +``mediagoblin_local.ini`` and restart MediaGoblin. + +Run + +.. code-block:: bash + + ./bin/gmg dbupdate + Now you should be able to submit videos, and mediagoblin should transcode them. @@ -92,7 +116,9 @@ To install these on Debianoid systems, run:: The ``scikits.audiolab`` package you will install in the next step depends on the ``libsndfile1-dev`` package, so we should install it. -On Debianoid systems, run:: +On Debianoid systems, run + +.. code-block:: bash sudo apt-get install libsndfile1-dev @@ -108,8 +134,15 @@ Then install ``scikits.audiolab`` for the spectrograms:: ./bin/pip install scikits.audiolab Add ``mediagoblin.media_types.audio`` to the ``media_types`` list in your -``mediagoblin_local.ini`` and restart MediaGoblin. You should now be able to -upload and listen to audio files! +``mediagoblin_local.ini`` and restart MediaGoblin. + +Run + +.. code-block:: bash + + ./bin/gmg dbupdate + +You should now be able to upload and listen to audio files! Ascii art @@ -117,7 +150,9 @@ Ascii art To enable ascii art support, first install the `chardet <http://pypi.python.org/pypi/chardet>`_ -library, which is necessary for creating thumbnails of ascii art:: +library, which is necessary for creating thumbnails of ascii art + +.. code-block:: bash ./bin/easy_install chardet @@ -131,4 +166,69 @@ the list would look like this:: media_types = mediagoblin.media_types.image, mediagoblin.media_types.ascii +Run + +.. code-block:: bash + + ./bin/gmg dbupdate + Now any .txt file you uploaded will be processed as ascii art! + + +STL / 3d model support +====================== + +To enable the "STL" 3d model support plugin, first make sure you have +a recentish `Blender <http://blender.org>`_ installed and available on +your execution path. This feature has been tested with Blender 2.63. +It may work on some earlier versions, but that is not guaranteed (and +is surely not to work prior to Blender 2.5X). + +Add ``mediagoblin.media_types.stl`` to the ``media_types`` list in your +``mediagoblin_local.ini`` and restart MediaGoblin. + +Run + +.. code-block:: bash + + ./bin/gmg dbupdate + +You should now be able to upload .obj and .stl files and MediaGoblin +will be able to present them to your wide audience of admirers! + +PDF and Document +================ + +To enable the "PDF and Document" support plugin, you need pdftocairo, pdfinfo, +unoconv with headless support. All executables must be on your execution path. + +To install this on Fedora: + +.. code-block:: bash + + sudo yum install -y poppler-utils unoconv libreoffice-headless + +pdf.js relies on git submodules, so be sure you have fetched them: + +.. code-block:: bash + + git submodule init + git submodule update + +This feature has been tested on Fedora with: + poppler-utils-0.20.2-9.fc18.x86_64 + unoconv-0.5-2.fc18.noarch + libreoffice-headless-3.6.5.2-8.fc18.x86_64 + +It may work on some earlier versions, but that is not guaranteed. + +Add ``mediagoblin.media_types.pdf`` to the ``media_types`` list in your +``mediagoblin_local.ini`` and restart MediaGoblin. + +Run + +.. code-block:: bash + + ./bin/gmg dbupdate + + diff --git a/docs/source/siteadmin/plugins.rst b/docs/source/siteadmin/plugins.rst index 75562d4b..baca381d 100644 --- a/docs/source/siteadmin/plugins.rst +++ b/docs/source/siteadmin/plugins.rst @@ -44,29 +44,33 @@ If the plugin is available on the `Python Package Index pip install <plugin-name> For example, if we wanted to install the plugin named -"mediagoblin-restrictfive", we would do:: +"mediagoblin-licenses" (which allows you to customize the licenses you +offer for your media), we would do:: - pip install mediagoblin-restrictfive + pip install mediagoblin-licenses .. Note:: If you're using a virtual environment, make sure to activate the - virtual environment before installing with pip. Otherwise the - plugin may get installed in a different environment than the one - MediaGoblin is installed in. + virtual environment before installing with pip. Otherwise the plugin + may get installed in a different environment than the one MediaGoblin + is installed in. Also make sure, you use e.g. pip-2.7 if your default + python (and thus pip) is python 3 (e.g. in Ubuntu). Once you've installed the plugin software, you need to tell MediaGoblin that this is a plugin you want MediaGoblin to use. To do that, you edit the ``mediagoblin.ini`` file and add the plugin as a subsection of the plugin section. -For example, say the "mediagoblin-restrictfive" plugin had the Python -package path ``restrictfive``, then you would add ``restrictfive`` to +For example, say the "mediagoblin-licenses" plugin has the Python +package path ``mediagoblin_licenses``, then you would add ``mediagoblin_licenses`` to the ``plugins`` section as a subsection:: [plugins] - [[restrictfive]] + [[mediagoblin_licenses]] + license_01=abbrev1, name1, http://url1 + license_02=abbrev2, name1, http://url2 Configuring plugins @@ -112,7 +116,7 @@ Removing plugins To remove a plugin, use ``pip uninstall``. For example:: - pip uninstall mediagoblin-restrictfive + pip uninstall mediagoblin-licenses .. Note:: diff --git a/docs/source/siteadmin/production-deployments.rst b/docs/source/siteadmin/production-deployments.rst index 356fab7f..1a32d95e 100644 --- a/docs/source/siteadmin/production-deployments.rst +++ b/docs/source/siteadmin/production-deployments.rst @@ -52,7 +52,7 @@ as the basis for your script: :: Separate Celery --------------- -While the ``./lazyserer.sh`` configuration provides an efficient way to +While the ``./lazyserver.sh`` configuration provides an efficient way to start using a MediaGoblin instance, it is not suitable for production deployments for several reasons: @@ -77,6 +77,17 @@ Modify your existing MediaGoblin and application init scripts, if necessary, to prevent them from starting their own ``celeryd`` processes. +.. _sentry: + +Set up sentry to monitor exceptions +----------------------------------- + +We have a plugin for `raven`_ integration, see the ":doc:`/plugindocs/raven`" +documentation. + +.. _`raven`: http://raven.readthedocs.org + + .. _init-script: Use an Init Script diff --git a/docs/source/siteadmin/relnotes.rst b/docs/source/siteadmin/relnotes.rst index 1b70b31c..04863ec6 100644 --- a/docs/source/siteadmin/relnotes.rst +++ b/docs/source/siteadmin/relnotes.rst @@ -19,6 +19,166 @@ This chapter has important information for releases in it. If you're upgrading from a previous release, please read it carefully, or at least skim over it. +0.3.3 +===== + +**Do this to upgrade** + +1. Make sure to run ``bin/gmg dbupdate`` after upgrading. +2. OpenStreetMap is now a plugin, so if you want to use it, add the + following to your config file: + + .. code-block:: ini + + [plugins] + [[mediagoblin.plugins.geolocation]] + +If you have your own theme, you may need to make some adjustments to +it as some theme related things may have changed in this release. If +you run into problems, don't hesitate to +`contact us <http://mediagoblin.org/pages/join.html>`_ +(IRC is often best). + +**New features** + +* New dropdown menu for accessing various features. + +* Significantly improved URL generation. Now mediagoblin won't give + up on making a slug if it looks like there will be a duplicate; + it'll try extra hard to generate a meaningful one instead. + + Similarly, linking to an id no longer can possibly conflict with + linking to a slug; /u/username/m/id:35/ is the kind of reference we + now use to linking to entries with ids. However, old links with + entries that linked to ids should work just fine with our migration. + The only urls that might break in this release are ones using colons + or equal signs. + +* New template hooks for plugin authoring. + +* As a demonstration of new template hooks for plugin authoring, + openstreetmap support now moved to a plugin! + +* Method to add media to collections switched from icon of paperclip + to button with "add to collection" text. + +* Bug where videos often failed to produce a proper thumbnail fixed! + +* Copying around files in MediaGoblin now much more efficient, doesn't + waste gobs of memory. + +* Video transcoding now optional for videos that meet certain + criteria. By default, MediaGoblin will not transcode webm videos + that are smaller in resolution than the MediaGoblin defaults, and + MediaGoblin can also be configured to allow theora files to not be + transcoded as well. + +* Per-user license preference option; always want your uploads to be + BY-SA and tired of changing that field? You can now set your + license preference in your user settings. + +* Video player now responsive; better for mobile! + +* You can now delete your account from the user preferences page if + you so wish. + +**Other changes** + +* Plugin writers: Internal restructuring led to mediagoblin.db.sql* be + mediagoblin.db.* starting from 0.3.3 + +* Dependency list has been reduced not requiring the "webob" package anymore. + +* And many small fixes/improvements, too numerous to list! + + +0.3.2 +===== + +This will be the last release that is capable of converting from an earlier +MongoDB-based MediaGoblin instance to the newer SQL-based system. + +**Do this to upgrade** + + # directory of your mediagoblin install + cd /srv/mediagoblin.example.org + + # copy source for this release + git fetch + git checkout tags/v0.3.2 + + # perform any needed database updates + bin/gmg dbupdate + + # restart your servers however you do that, e.g., + sudo service mediagoblin-paster restart + sudo service mediagoblin-celeryd restart + + +**New features** + +* **3d model support!** + + You can now upload STL and OBJ files and display them in + MediaGoblin. Requires a recent-ish Blender; for details see: + :ref:`deploying-chapter` + +* **trim_whitespace** + + We bundle the optional plugin trim_whitespace which reduces the size + of the delivered html output by reducing redundant whitespace. + + See :ref:`core-plugin-section` for plugin documentation + +* **A new API!** + + It isn't well documented yet but we do have an API. There is an + `android application in progress <https://gitorious.org/mediagoblin/mediagoblin-android>`_ + which makes use of it, and there are some demo applications between + `automgtic <https://github.com/jwandborg/automgtic>`_, an + automatic media uploader for your desktop + and `OMGMG <https://github.com/jwandborg/omgmg>`_, an example of + a web application hooking up to the API. + + This is a plugin, so you have to enable it in your mediagoblin + config file by adding a section under [plugins] like:: + + [plugins] + [[mediagoblin.plugins.api]] + + Note that the API works but is not nailed down... the way it is + called may change in future releases. + +* **OAuth login support** + + For applications that use OAuth to connect to the API. + + This is a plugin, so you have to enable it in your mediagoblin + config file by adding a section under [plugins] like:: + + [plugins] + [[mediagoblin.plugins.oauth]] + +* **Collections** + + We now have user-curated collections support. These are arbitrary + galleries that are customizable by users. You can add media to + these by clicking on the paperclip icon when logged in and looking + at a media entry. + +* **OpenStreetMap licensing display improvements** + + More accurate display of OSM licensing, and less disruptive: you + click to "expand" the display of said licensing. + + Geolocation is also now on by default. + +* **Miscelaneous visual improvements** + + We've made a number of small visual improvements including newer and + nicer looking thumbnails and improved checkbox placement. + + 0.3.1 ===== |