diff options
Diffstat (limited to 'docs/source')
| -rw-r--r-- | docs/source/devel/migrations.rst | 62 | ||||
| -rw-r--r-- | docs/source/index.rst | 3 | ||||
| -rw-r--r-- | docs/source/pluginwriter/api.rst | 274 | ||||
| -rw-r--r-- | docs/source/pluginwriter/database.rst | 9 | ||||
| -rw-r--r-- | docs/source/pluginwriter/media_type_hooks.rst | 38 | ||||
| -rw-r--r-- | docs/source/pluginwriter/quickstart.rst | 6 | ||||
| -rw-r--r-- | docs/source/pluginwriter/tests.rst | 64 | ||||
| -rw-r--r-- | docs/source/siteadmin/deploying.rst | 44 | ||||
| -rw-r--r-- | docs/source/siteadmin/media-types.rst | 52 | ||||
| -rw-r--r-- | docs/source/siteadmin/production-deployments.rst | 64 | ||||
| -rw-r--r-- | docs/source/siteadmin/relnotes.rst | 78 | ||||
| -rw-r--r-- | docs/source/siteadmin/theming.rst | 2 |
12 files changed, 636 insertions, 60 deletions
diff --git a/docs/source/devel/migrations.rst b/docs/source/devel/migrations.rst new file mode 100644 index 00000000..16c02b04 --- /dev/null +++ b/docs/source/devel/migrations.rst @@ -0,0 +1,62 @@ +.. MediaGoblin Documentation + + Written in 2011, 2012 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + +========== +Migrations +========== + +So, about migrations. Every time we change the way the database +structure works, we need to add a migration so that people running +older codebases can have their databases updated to the new structure +when they run `./bin/gmg dbupdate`. + +The first time `./bin/gmg dbupdate` is run by a user, it creates the +tables at the current state that they're defined in models.py and sets +the migration number to the current migration... after all, migrations +only exist to get things to the current state of the db. After that, +every migration is run with dbupdate. + +There's a few things you need to know: + +- We use `sqlalchemy-migrate + <http://code.google.com/p/sqlalchemy-migrate/>`_. + See `their docs <https://sqlalchemy-migrate.readthedocs.org/>`_. +- `Alembic <https://bitbucket.org/zzzeek/alembic>`_ might be a better + choice than sqlalchemy-migrate now or in the future, but we + originally decided not to use it because it didn't have sqlite + support. It's not clear if that's changed. +- SQLAlchemy has two parts to it, the ORM and the "core" interface. + We DO NOT use the ORM when running migrations. Think about it: the + ORM is set up with an expectation that the models already reflect a + certain pattern. But if a person is moving from their old patern + and are running tools to *get to* the current pattern, of course + their current database structure doesn't match the state of the ORM! +- How to write migrations? Maybe there will be a tutorial here in the + future... in the meanwhile, look at existing migrations in + `mediagoblin/db/migrations.py` and look in + `mediagoblin/tests/test_sql_migrations.py` for examples. +- Common pattern: use `inspect_table` to get the current state + of the table before we run alterations on it. +- Make sure you set the RegisterMigration to be the next migration in + order. +- What happens if you're adding a *totally new* table? In this case, + you should copy the table in entirety as it exists into + migrations.py then create the tables based off of that... see + add_collection_tables. This is easier than reproducing the SQL by + hand. +- If you're writing a feature branch, you don't need to keep adding + migrations every time you change things around if your database + structure is in flux. Just alter your migrations so that they're + correct for the merge into master. + +That's it for now! Good luck! diff --git a/docs/source/index.rst b/docs/source/index.rst index 7f692d57..de6c9c0d 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -73,6 +73,8 @@ This guide covers writing new GNU MediaGoblin plugins. pluginwriter/quickstart pluginwriter/database pluginwriter/api + pluginwriter/tests + pluginwriter/media_type_hooks Part 4: Developer's Zone @@ -86,6 +88,7 @@ This chapter contains various information for developers. devel/codebase devel/storage devel/originaldesigndecisions + devel/migrations Indices and tables diff --git a/docs/source/pluginwriter/api.rst b/docs/source/pluginwriter/api.rst index df933511..29adb691 100644 --- a/docs/source/pluginwriter/api.rst +++ b/docs/source/pluginwriter/api.rst @@ -11,6 +11,7 @@ Dedication along with this software. If not, see <http://creativecommons.org/publicdomain/zero/1.0/>. +.. _plugin-api-chapter: ========== Plugin API @@ -23,7 +24,27 @@ Authors are encouraged to develop plugins and work with the MediaGoblin community to keep them up to date, but this API will be a moving target for a few releases. -Please check the release notes for updates! +Please check the :ref:`release-notes` for updates! + + +How are hooks added? Where do I find them? +------------------------------------------- + +Much of this document talks about hooks, both as in terms of regular +hooks and template hooks. But where do they come from, and how can +you find a list of them? + +For the moment, the best way to find available hooks is to check the +source code itself. (Yes, we should start a more official hook +listing with descriptions soon.) But many hooks you may need do not +exist yet: what to do then? + +The plan at present is that we are adding hooks as people need them, +with community discussion. If you find that you need a hook and +MediaGoblin at present doesn't provide it at present, please +`http://mediagoblin.org/pages/join.html <talk to us>`_! We'll +evaluate what to do from there. + :mod:`pluginapi` Module ----------------------- @@ -48,3 +69,254 @@ example might look like:: This means that when people enable your plugin in their config you'll be able to provide defaults as well as type validation. +You can access this via the app_config variables in mg_globals, or you +can use a shortcut to get your plugin's config section:: + + >>> from mediagoblin.tools import pluginapi + # Replace with the path to your plugin. + # (If an external package, it won't be part of mediagoblin.plugins) + >>> floobie_config = pluginapi.get_config('mediagoblin.plugins.floobifier') + >>> floobie_dir = floobie_config['floobie_dir'] + # This is the same as the above + >>> from mediagoblin import mg_globals + >>> config = mg_globals.global_config['plugins']['mediagoblin.plugins.floobifier'] + >>> floobie_dir = floobie_config['floobie_dir'] + +A tip: you have access to the `%(here)s` variable in your config, +which is the directory that the user's mediagoblin config is running +out of. So for example, your plugin may need a "floobie" directory to +store floobs in. You could give them a reasonable default that makes +use of the default `user_dev` location, but allow users to override +it, like so:: + + [plugin_spec] + floobie_dir = string(default="%(here)s/user_dev/floobs/") + +Note, this is relative to the user's mediagoblin config directory, +*not* your plugin directory! + + +Context Hooks +------------- + +View specific hooks ++++++++++++++++++++ + +You can hook up to almost any template called by any specific view +fairly easily. As long as the view directly or indirectly uses the +method ``render_to_response`` you can access the context via a hook +that has a key in the format of the tuple:: + + (view_symbolic_name, view_template_path) + +Where the "view symbolic name" is the same parameter used in +``request.urlgen()`` to look up the view. So say we're wanting to add +something to the context of the user's homepage. We look in +mediagoblin/user_pages/routing.py and see:: + + add_route('mediagoblin.user_pages.user_home', + '/u/<string:user>/', + 'mediagoblin.user_pages.views:user_home') + +Aha! That means that the name is ``mediagoblin.user_pages.user_home``. +Okay, so then we look at the view at the +``mediagoblin.user_pages.user_home`` method:: + + @uses_pagination + def user_home(request, page): + # [...] whole bunch of stuff here + return render_to_response( + request, + 'mediagoblin/user_pages/user.html', + {'user': user, + 'user_gallery_url': user_gallery_url, + 'media_entries': media_entries, + 'pagination': pagination}) + +Nice! So the template appears to be +``mediagoblin/user_pages/user.html``. Cool, that means that the key +is:: + + ("mediagoblin.user_pages.user_home", + "mediagoblin/user_pages/user.html") + +The context hook uses ``hook_transform()`` so that means that if we're +hooking into it, our hook will both accept one argument, ``context``, +and should return that modified object, like so:: + + def add_to_user_home_context(context): + context['foo'] = 'bar' + return context + + hooks = { + ("mediagoblin.user_pages.user_home", + "mediagoblin/user_pages/user.html"): add_to_user_home_context} + + +Global context hooks +++++++++++++++++++++ + +If you need to add something to the context of *every* view, it is not +hard; there are two hooks hook that also uses hook_transform (like the +above) but make available what you are providing to *every* view. + +Note that there is a slight, but critical, difference between the two. + +The most general one is the ``'template_global_context'`` hook. This +one is run only once, and is read into the global context... all views +will get access to what are in this dict. + +The slightly more expensive but more powerful one is +``'template_context_prerender'``. This one is not added to the global +context... it is added to the actual context of each individual +template render right before it is run! Because of this you also can +do some powerful and crazy things, such as checking the request object +or other parts of the context before passing them on. + + +Adding static resources +----------------------- + +It's possible to add static resources for your plugin. Say your +plugin needs some special javascript and images... how to provide +them? Then how to access them? MediaGoblin has a way! + + +Attaching to the hook ++++++++++++++++++++++ + +First, you need to register your plugin's resources with the hook. +This is pretty easy actually: you just need to provide a function that +passes back a PluginStatic object. + +.. autoclass:: mediagoblin.tools.staticdirect.PluginStatic + + +Running plugin assetlink +++++++++++++++++++++++++ + +In order for your plugin assets to be properly served by MediaGoblin, +your plugin's asset directory needs to be symlinked into the directory +that plugin assets are served from. To set this up, run:: + + ./bin/gmg assetlink + + +Using staticdirect +++++++++++++++++++ + +Once you have this, you will want to be able to of course link to your +assets! MediaGoblin has a "staticdirect" tool; you want to use this +like so in your templates:: + + staticdirect("css/monkeys.css", "mystaticname") + +Replace "mystaticname" with the name you passed to PluginStatic. The +staticdirect method is, for convenience, attached to the request +object, so you can access this in your templates like: + +.. code-block:: html + + <img alt="A funny bunny" + src="{{ request.staticdirect('images/funnybunny.png', 'mystaticname') }}" /> + + +Additional hook tips +-------------------- + +This section aims to explain some tips in regards to adding hooks to +the MediaGoblin repository. + +WTForms hooks ++++++++++++++ + +We haven't totally settled on a way to tranform wtforms form objects, +but here's one way. In your view:: + + from mediagoblin.foo.forms import SomeForm + + def some_view(request) + form_class = hook_transform('some_form_transform', SomeForm) + form = form_class(request.form) + +Then to hook into this form, do something in your plugin like:: + + import wtforms + + class SomeFormAdditions(wtforms.Form): + new_datefield = wtforms.DateField() + + def transform_some_form(orig_form): + class ModifiedForm(orig_form, SomeFormAdditions) + return ModifiedForm + + hooks = { + 'some_form_transform': transform_some_form} + + +Interfaces +++++++++++ + +If you want to add a pseudo-interface, it's not difficult to do so. +Just write the interface like so:: + + class FrobInterface(object): + """ + Interface for Frobbing. + + Classes implementing this interface should provide defrob and frob. + They may also implement double_frob, but it is not required; if + not provided, we will use a general technique. + """ + + def defrob(self, frobbed_obj): + """ + Take a frobbed_obj and defrob it. Returns the defrobbed object. + """ + raise NotImplementedError() + + def frob(self, normal_obj): + """ + Take a normal object and frob it. Returns the frobbed object. + """ + raise NotImplementedError() + + def double_frob(self, normal_obj): + """ + Frob this object and return it multiplied by two. + """ + return self.frob(normal_obj) * 2 + + + def some_frob_using_method(): + # something something something + frobber = hook_handle(FrobInterface) + frobber.frob(blah) + + # alternately you could have a default + frobber = hook_handle(FrobInterface) or DefaultFrobber + frobber.defrob(foo) + + +It's fine to use your interface as the key instead of a string if you +like. (Usually this is messy, but since interfaces are public and +since you need to import them into your plugin anyway, interfaces +might as well be keys.) + +Then a plugin providing your interface can be like:: + + from mediagoblin.foo.frobfrogs import FrobInterface + from frogfrobber import utils + + class FrogFrobber(FrobInterface): + """ + Takes a frogputer science approach to frobbing. + """ + def defrob(self, frobbed_obj): + return utils.frog_defrob(frobbed_obj) + + def frob(self, normal_obj): + return utils.frog_frob(normal_obj) + + hooks = { + FrobInterface: lambda: return FrogFrobber} diff --git a/docs/source/pluginwriter/database.rst b/docs/source/pluginwriter/database.rst index 58edf3a0..603a19eb 100644 --- a/docs/source/pluginwriter/database.rst +++ b/docs/source/pluginwriter/database.rst @@ -12,9 +12,12 @@ <http://creativecommons.org/publicdomain/zero/1.0/>. -======== -Database -======== +.. _plugin-database-chapter: + + +=========================== +Database models for plugins +=========================== Accessing Existing Data diff --git a/docs/source/pluginwriter/media_type_hooks.rst b/docs/source/pluginwriter/media_type_hooks.rst new file mode 100644 index 00000000..498b0b54 --- /dev/null +++ b/docs/source/pluginwriter/media_type_hooks.rst @@ -0,0 +1,38 @@ +================== + Media Type hooks +================== + +This documents the hooks that are currently available for ``media_type`` plugins. + +What hooks are available? +========================= + +'sniff_handler' +--------------- + +This hook is used by ``sniff_media`` in ``mediagoblin.media_types.__init__``. +Your media type should return its ``sniff_media`` method when this hook is +called. + +.. Note:: + Your ``sniff_media`` method should return either the ``media_type`` or + ``None``. + +'get_media_type_and_manager' +---------------------------- + +This hook is used by ``get_media_type_and_manager`` in +``mediagoblin.media_types.__init__``. When this hook is called, your media type +plugin should check if it can handle the given extension. If so, your media +type plugin should return the media type and media manager. + +('media_manager', MEDIA_TYPE) +----------------------------- + +If you already know the string representing the media type of a type +of media, you can pull down the manager specifically. Note that this +hook is not a string but a tuple of two strings, the latter being the +name of the media type. + +This is used by media entries to pull down their media managers, and +so on. diff --git a/docs/source/pluginwriter/quickstart.rst b/docs/source/pluginwriter/quickstart.rst index b5a63f79..6d45ea36 100644 --- a/docs/source/pluginwriter/quickstart.rst +++ b/docs/source/pluginwriter/quickstart.rst @@ -178,8 +178,10 @@ That's it for the quick start! Where to go from here ===================== -See the documentation on the plugin API for code samples and other -things you can use when building your plugin. +See the documentation on the :ref:`plugin-api-chapter` for code +samples and other things you can use when building your plugin. If +your plugin needs its own database models, see +:ref:`plugin-database-chapter`. See `Hitchhiker's Guide to Packaging <http://guide.python-distribute.org/>`_ for more information on diff --git a/docs/source/pluginwriter/tests.rst b/docs/source/pluginwriter/tests.rst new file mode 100644 index 00000000..fe99688f --- /dev/null +++ b/docs/source/pluginwriter/tests.rst @@ -0,0 +1,64 @@ +.. MediaGoblin Documentation + + Written in 2013 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + +============================== +Writing unit tests for plugins +============================== + +Here's a brief guide to writing unit tests for plugins. However, it +isn't really ideal. It also hasn't been well tested... yes, there's +some irony there :) + +Some notes: we're using py.test and webtest for unit testing stuff. +Keep that in mind. + +My suggestion is to mime the behavior of `mediagoblin/tests/` and put +that in your own plugin, like `myplugin/tests/`. Copy over +`conftest.py` and `pytest.ini` to your tests directory, but possibly +change the `test_app` fixture to match your own tests' config needs. +For example:: + + import pkg_resources + # [...] + + @pytest.fixture() + def test_app(request): + return get_app( + request, + mgoblin_config=pkg_resources.resource_filename( + 'myplugin.tests', 'myplugin_mediagoblin.ini')) + +In any test module in your tests directory you can then do:: + + def test_somethingorother(test_app): + # real code goes here + pass + +And you'll get a mediagoblin application wrapped in webtest passed in +to your environment. + +If your plugin needs to define multiple configuration setups, you can +actually set up multiple fixtures very easily for this. You can just +set up multiple fixtures with different names that point to different +configs and pass them in as that named argument. + +To run the tests, from mediagoblin's directory (make sure that your +plugin has been added to your mediagoblin checkout's virtualenv!) do:: + + ./runtests.sh /path/to/myplugin/tests/ + +replacing `/path/to/myplugin/` with the actual path to your plugin. + +NOTE: again, the above is untested, but it should probably work. If +you run into trouble, `contact us +<http://mediagoblin.org/pages/join.html>`_, preferably on IRC! diff --git a/docs/source/siteadmin/deploying.rst b/docs/source/siteadmin/deploying.rst index 326d2243..a892bbd4 100644 --- a/docs/source/siteadmin/deploying.rst +++ b/docs/source/siteadmin/deploying.rst @@ -197,13 +197,14 @@ Change to the MediaGoblin directory that you just created:: cd /srv/mediagoblin.example.org -Clone the MediaGoblin repository:: +Clone the MediaGoblin repository and set up the git submodules:: git clone git://gitorious.org/mediagoblin/mediagoblin.git + cd mediagoblin + git submodule init && git submodule update Set up the in-package virtualenv:: - cd mediagoblin (virtualenv --system-site-packages . || virtualenv .) && ./bin/python setup.py develop .. note:: @@ -234,7 +235,7 @@ This concludes the initial configuration of the development environment. In the future, when you update your codebase, you should also run:: - ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate + ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate && git submodule fetch Note: If you are running an active site, depending on your server configuration, you may need to stop it first or the dbupdate command @@ -245,6 +246,23 @@ update) Deploy MediaGoblin Services --------------------------- +Edit site configuration +~~~~~~~~~~~~~~~~~~~~~~~ + +A few basic properties must be set before MediaGoblin will work. First +make a copy of ``mediagoblin.ini`` for editing so the original config +file isn't lost:: + + cp mediagoblin.ini mediagoblin_local.ini + +Then: + - Set ``email_sender_address`` to the address you wish to be used as + the sender for system-generated emails + - Edit ``direct_remote_path``, ``base_dir``, and ``base_url`` if + your mediagoblin directory is not the root directory of your + vhost. + + Configure MediaGoblin to use the PostgreSQL database ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -280,11 +298,11 @@ browser to confirm that the service is operable. .. _webserver-config: -Connect the Webserver to MediaGoblin with FastCGI -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This section describes how to configure MediaGoblin to work via -FastCGI. Our configuration example will use nginx, however, you may +FastCGI and nginx +~~~~~~~~~~~~~~~~~ + +This configuration example will use nginx, however, you may use any webserver of your choice as long as it supports the FastCGI protocol. If you do not already have a web server, consider nginx, as the configuration files may be more clear than the @@ -350,6 +368,11 @@ this ``nginx.conf`` file should be modeled on the following:: alias /srv/mediagoblin.example.org/mediagoblin/user_dev/theme_static/; } + # Plugin static files (usually symlinked in) + location /plugin_static/ { + alias /srv/mediagoblin.example.org/mediagoblin/user_dev/plugin_static/; + } + # Mounting MediaGoblin itself via FastCGI. location / { fastcgi_pass 127.0.0.1:26543; @@ -385,6 +408,13 @@ Visit the site you've set up in your browser by visiting smaller deployments. However, for larger production deployments with larger processing requirements, see the ":doc:`production-deployments`" documentation. + + +Apache +~~~~~~ + +Instructions and scripts for running MediaGoblin on an Apache server +can be found on the `MediaGoblin wiki <http://wiki.mediagoblin.org/Deployment>`_. Security Considerations diff --git a/docs/source/siteadmin/media-types.rst b/docs/source/siteadmin/media-types.rst index 210094b9..3e8a94e9 100644 --- a/docs/source/siteadmin/media-types.rst +++ b/docs/source/siteadmin/media-types.rst @@ -18,16 +18,18 @@ Media Types ==================== In the future, there will be all sorts of media types you can enable, -but in the meanwhile there are three additional media types: video, audio -and ascii art. +but in the meanwhile there are five additional media types: video, audio, +ascii art, STL/3d models, PDF and Document. First, you should probably read ":doc:`configuration`" to make sure you know how to modify the mediagoblin config file. - Enabling Media Types ==================== +.. note:: + Media types are now plugins + Media types are enabled in your mediagoblin configuration file, typically it is created by copying ``mediagoblin.ini`` to ``mediagoblin_local.ini`` and then applying your changes to ``mediagoblin_local.ini``. If you don't already have a @@ -37,11 +39,13 @@ Most media types have additional dependencies that you will have to install. You will find descriptions on how to satisfy the requirements of each media type on this page. -To enable a media type, edit the ``media_types`` list in your -``mediagoblin_local.ini``. For example, if your system supported image and -video media types, then the list would look like this:: +To enable a media type, add the the media type under the ``[plugins]`` section +in you ``mediagoblin_local.ini``. For example, if your system supported image +and video media types, then it would look like this:: - media_types = mediagoblin.media_types.image, mediagoblin.media_types.video + [plugins] + [[mediagoblin.media_types.image]] + [[mediagoblin.media_types.video]] Note that after enabling new media types, you must run dbupdate like so:: @@ -83,8 +87,8 @@ good/bad/ugly). On Debianoid systems gstreamer0.10-ffmpeg -Add ``mediagoblin.media_types.video`` to the ``media_types`` list in your -``mediagoblin_local.ini`` and restart MediaGoblin. +Add ``[[mediagoblin.media_types.video]]`` under the ``[plugins]`` section in +your ``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -133,7 +137,7 @@ Then install ``scikits.audiolab`` for the spectrograms:: ./bin/pip install scikits.audiolab -Add ``mediagoblin.media_types.audio`` to the ``media_types`` list in your +Add ``[[mediagoblin.media_types.audio]]`` under the ``[plugins]`` section in your ``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -158,13 +162,8 @@ library, which is necessary for creating thumbnails of ascii art Next, modify (and possibly copy over from ``mediagoblin.ini``) your -``mediagoblin_local.ini``. In the ``[mediagoblin]`` section, add -``mediagoblin.media_types.ascii`` to the ``media_types`` list. - -For example, if your system supported image and ascii art media types, then -the list would look like this:: - - media_types = mediagoblin.media_types.image, mediagoblin.media_types.ascii +``mediagoblin_local.ini``. In the ``[plugins]`` section, add +``[[mediagoblin.media_types.ascii]]``. Run @@ -184,7 +183,7 @@ your execution path. This feature has been tested with Blender 2.63. It may work on some earlier versions, but that is not guaranteed (and is surely not to work prior to Blender 2.5X). -Add ``mediagoblin.media_types.stl`` to the ``media_types`` list in your +Add ``[[mediagoblin.media_types.stl]]`` under the ``[plugins]`` section in your ``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -199,8 +198,16 @@ will be able to present them to your wide audience of admirers! PDF and Document ================ -To enable the "PDF and Document" support plugin, you need pdftocairo, pdfinfo, -unoconv with headless support. All executables must be on your execution path. +To enable the "PDF and Document" support plugin, you need: + +1. pdftocairo and pdfinfo for pdf only support. + +2. unoconv with headless support to support converting libreoffice supported + documents as well, such as doc/ppt/xls/odf/odg/odp and more. + For the full list see mediagoblin/media_types/pdf/processing.py, + unoconv_supported. + +All executables must be on your execution path. To install this on Fedora: @@ -208,6 +215,9 @@ To install this on Fedora: sudo yum install -y poppler-utils unoconv libreoffice-headless +Note: You can leave out unoconv and libreoffice-headless if you want only pdf +support. This will result in a much smaller list of dependencies. + pdf.js relies on git submodules, so be sure you have fetched them: .. code-block:: bash @@ -222,7 +232,7 @@ This feature has been tested on Fedora with: It may work on some earlier versions, but that is not guaranteed. -Add ``mediagoblin.media_types.pdf`` to the ``media_types`` list in your +Add ``[[mediagoblin.media_types.pdf]]`` under the ``[plugins]`` section in your ``mediagoblin_local.ini`` and restart MediaGoblin. Run diff --git a/docs/source/siteadmin/production-deployments.rst b/docs/source/siteadmin/production-deployments.rst index 1a32d95e..839d3ce5 100644 --- a/docs/source/siteadmin/production-deployments.rst +++ b/docs/source/siteadmin/production-deployments.rst @@ -22,12 +22,14 @@ MediaGoblin in actual production environments. Consider Deploy with Paste ----------------- -The instance configured with ``./lazyserver.sh`` is not ideal for a -production MediaGoblin deployment. Ideally, you should be able to use -an "init" or "control" script to launch and restart the MediaGoblin +The MediaGoblin WSGI application instance you get with ``./lazyserver.sh`` is +not ideal for a production MediaGoblin deployment. Ideally, you should be able +to use an "init" or "control" script to launch and restart the MediaGoblin process. -Use the following command as the basis for such a script: :: +Use the following command as the basis for such a script: + +.. code-block:: bash CELERY_ALWAYS_EAGER=true \ /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ @@ -41,7 +43,9 @@ synchronously, and the user will advance to the next page only after processing is complete. If we take Celery out of "always eager mode," the user will be able to immediately return to the MediaGoblin site while processing is ongoing. In these cases, use the following command -as the basis for your script: :: +as the basis for your script: + +.. code-block:: bash CELERY_ALWAYS_EAGER=false \ /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ @@ -52,30 +56,40 @@ as the basis for your script: :: Separate Celery --------------- -While the ``./lazyserver.sh`` configuration provides an efficient way to -start using a MediaGoblin instance, it is not suitable for production -deployments for several reasons: +MediaGoblin uses `Celery`_ to handle heavy and long-running tasks. Celery can +be launched in two ways: + +1. Embedded in the MediaGoblin WSGI application [#f-mediagoblin-wsgi-app]_. + This is the way ``./lazyserver.sh`` does it for you. It's simple as you + only have to run one process. The only bad thing with this is that the + heavy and long-running tasks will run *in* the webserver, keeping the user + waiting each time some heavy lifting is needed as in for example processing + a video. This could lead to problems as an aborted connection will halt any + processing and since most front-end web servers *will* terminate your + connection if it doesn't get any response from the MediaGoblin WSGI + application in a while. + +2. As a separate process communicating with the MediaGoblin WSGI application + via a `broker`_. This offloads the heavy lifting from the MediaGoblin WSGI + application and users will be able to continue to browse the site while the + media is being processed in the background. + +.. _`broker`: http://docs.celeryproject.org/en/latest/getting-started/brokers/ +.. _`celery`: http://www.celeryproject.org/ + -In nearly every scenario, work on the Celery queue will need to -balance with the demands of other processes, and cannot proceed -synchronously. This is a particularly relevant problem if you use -MediaGoblin to host video content. Processing with Celery ought to be -operationally separate from the MediaGoblin application itself, this -simplifies management and support better workload distribution. +.. [#f-mediagoblin-wsgi-app] The MediaGoblin WSGI application is the part that + of MediaGoblin that processes HTTP requests. -Basically, if you're doing anything beyond a trivial workload, such as -image hosting for a small set of users, or have limited media types -such as "ASCII art" or icon sharing, you will need to run ``celeryd`` -as a separate process. +To launch Celery separately from the MediaGoblin WSGI application: -Build an :ref:`init script <init-script>` around the following -command:: +1. Make sure that the ``CELERY_ALWAYS_EAGER`` environment variable is unset or + set to ``false`` when launching the MediaGoblin WSGI application. +2. Start the ``celeryd`` main process with - CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd + .. code-block:: bash -Modify your existing MediaGoblin and application init scripts, if -necessary, to prevent them from starting their own ``celeryd`` -processes. + CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd .. _sentry: @@ -102,7 +116,7 @@ These are scripts provided by the MediaGoblin community: Debian * `GNU MediaGoblin init scripts - <https://github.com/jwandborg/mediagoblin-init-scripts>`_ + <https://github.com/joar/mediagoblin-init-scripts>`_ by `Joar Wandborg <http://wandborg.se>`_ Arch Linux diff --git a/docs/source/siteadmin/relnotes.rst b/docs/source/siteadmin/relnotes.rst index 04863ec6..35415b66 100644 --- a/docs/source/siteadmin/relnotes.rst +++ b/docs/source/siteadmin/relnotes.rst @@ -11,6 +11,8 @@ Dedication along with this software. If not, see <http://creativecommons.org/publicdomain/zero/1.0/>. +.. _release-notes: + ============= Release Notes ============= @@ -19,6 +21,82 @@ This chapter has important information for releases in it. If you're upgrading from a previous release, please read it carefully, or at least skim over it. + +0.4.1 +===== + +This is a bugfix release for 0.4.0. This only implements one major +fix in the newly released document support which prevented the +"conversion via libreoffice" feature. + +If you were running 0.4.0 you can upgrade to v0.4.1 via a simple +switch and restarting mediagoblin/celery with no other actions. + +Otherwise, follow 0.4.0 instructions. + + +0.4.0 +===== + +**Do this to upgrade** + +1. Make sure to run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + after upgrading. +2. See "For Theme authors" if you have a custom theme. +3. Note that ``./bin/gmg theme assetlink`` is now just + ``./bin/gmg assetlink`` and covers both plugins and assets. + Keep on reading to hear more about new plugin features. +4. If you want to take advantage of new plugins that have statically + served assets, you are going to need to add the new "plugin_static" + section to your nginx config. Basically the following for nginx:: + + # Plugin static files (usually symlinked in) + location /plugin_static/ { + alias /srv/mediagoblin.example.org/mediagoblin/user_dev/plugin_static/; + } + + Similarly, if you've got a modified paste config, you may want to + borrow the app:plugin_static section from the default paste.ini + file. +5. We now use itsdangerous for sessions; if you had any references to + beaker in your paste config you can remove them. Again, see the + default paste.ini config +6. We also now use git submodules. Please do: + ``git submodule init && git submodule update`` + You will need to do this to use the new PDF support. + +**For theme authors** + +If you have your own theme or you have any "user modified templates", +please note the following: + +* mediagoblin/bits/ files above-content.html, body-end.html, + body-start.html now are renamed... they have underscores instead of + dashes in the filenames now :) +* There's a new file: ``mediagoblin/bits/frontpage_welcome.html``. + You can easily customize this to give a welcome page appropriate to + your site. + + +**New features** +* PDF media type! +* Improved plugin system. More flexible, better documented, with a + new plugin authoring section of the docs. +* itsdangerous based sessions. No more beaker! +* New, experimental Piwigo-based API. This means you should be able + to use MediaGoblin with something like Shotwell. (Again, a word of + caution: this is *very experimental*!) +* Human readable timestamps, and the option to display the original + date of an image when available (available as the + "original_date_visible" variable) +* Moved unit testing system from nosetests to py.test so we can better + handle issues with sqlalchemy exploding with different database + configurations. Long story :) +* You can now disable the ability to post comments. +* Tags now can be up to length 255 characters by default. + + 0.3.3 ===== diff --git a/docs/source/siteadmin/theming.rst b/docs/source/siteadmin/theming.rst index 98ec6de7..11ae3875 100644 --- a/docs/source/siteadmin/theming.rst +++ b/docs/source/siteadmin/theming.rst @@ -51,7 +51,7 @@ want to install this theme! Don't worry, it's fairly painless. 5. Link the assets so that they can be served by your web server:: - $ ./bin/gmg theme assetlink + $ ./bin/gmg assetlink .. Note:: |