7 files changed, 207 insertions, 25 deletions
diff --git a/docs/source/devel/migrations.rst b/docs/source/devel/migrations.rst
new file mode 100644
index 00000000..16c02b04
--- /dev/null
+++ b/docs/source/devel/migrations.rst
@@ -0,0 +1,62 @@
+.. MediaGoblin Documentation
+
+   Written in 2011, 2012 by MediaGoblin contributors
+
+   To the extent possible under law, the author(s) have dedicated all
+   copyright and related and neighboring rights to this software to
+   the public domain worldwide. This software is distributed without
+   any warranty.
+
+   You should have received a copy of the CC0 Public Domain
+   Dedication along with this software. If not, see
+   <http://creativecommons.org/publicdomain/zero/1.0/>.
+
+==========
+Migrations
+==========
+
+So, about migrations.  Every time we change the way the database
+structure works, we need to add a migration so that people running
+older codebases can have their databases updated to the new structure
+when they run `./bin/gmg dbupdate`.
+
+The first time `./bin/gmg dbupdate` is run by a user, it creates the
+tables at the current state that they're defined in models.py and sets
+the migration number to the current migration... after all, migrations
+only exist to get things to the current state of the db.  After that,
+every migration is run with dbupdate.
+
+There's a few things you need to know:
+
+- We use `sqlalchemy-migrate
+  <http://code.google.com/p/sqlalchemy-migrate/>`_.
+  See `their docs <https://sqlalchemy-migrate.readthedocs.org/>`_.
+- `Alembic <https://bitbucket.org/zzzeek/alembic>`_ might be a better
+  choice than sqlalchemy-migrate now or in the future, but we
+  originally decided not to use it because it didn't have sqlite
+  support.  It's not clear if that's changed.
+- SQLAlchemy has two parts to it, the ORM and the "core" interface.
+  We DO NOT use the ORM when running migrations.  Think about it: the
+  ORM is set up with an expectation that the models already reflect a
+  certain pattern.  But if a person is moving from their old patern
+  and are running tools to *get to* the current pattern, of course
+  their current database structure doesn't match the state of the ORM!
+- How to write migrations?  Maybe there will be a tutorial here in the
+  future... in the meanwhile, look at existing migrations in
+  `mediagoblin/db/migrations.py` and look in
+  `mediagoblin/tests/test_sql_migrations.py` for examples.
+- Common pattern: use `inspect_table` to get the current state
+  of the table before we run alterations on it.
+- Make sure you set the RegisterMigration to be the next migration in
+  order.
+- What happens if you're adding a *totally new* table?  In this case,
+  you should copy the table in entirety as it exists into
+  migrations.py then create the tables based off of that... see
+  add_collection_tables.  This is easier than reproducing the SQL by
+  hand.
+- If you're writing a feature branch, you don't need to keep adding
+  migrations every time you change things around if your database
+  structure is in flux.  Just alter your migrations so that they're
+  correct for the merge into master.
+
+That's it for now!  Good luck!
diff --git a/docs/source/index.rst b/docs/source/index.rst
index d71f39f8..de6c9c0d 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -74,6 +74,7 @@ This guide covers writing new GNU MediaGoblin plugins.
    pluginwriter/database
    pluginwriter/api
    pluginwriter/tests
+   pluginwriter/media_type_hooks
 
 
 Part 4: Developer's Zone
@@ -87,6 +88,7 @@ This chapter contains various information for developers.
    devel/codebase
    devel/storage
    devel/originaldesigndecisions
+   devel/migrations
 
 
 Indices and tables
diff --git a/docs/source/pluginwriter/api.rst b/docs/source/pluginwriter/api.rst
index 66def173..29adb691 100644
--- a/docs/source/pluginwriter/api.rst
+++ b/docs/source/pluginwriter/api.rst
@@ -69,6 +69,32 @@ example might look like::
 This means that when people enable your plugin in their config you'll
 be able to provide defaults as well as type validation.
 
+You can access this via the app_config variables in mg_globals, or you
+can use a shortcut to get your plugin's config section::
+
+  >>> from mediagoblin.tools import pluginapi
+  # Replace with the path to your plugin.
+  # (If an external package,  it won't be part of mediagoblin.plugins)
+  >>> floobie_config = pluginapi.get_config('mediagoblin.plugins.floobifier')
+  >>> floobie_dir = floobie_config['floobie_dir']
+  # This is the same as the above
+  >>> from mediagoblin import mg_globals
+  >>> config = mg_globals.global_config['plugins']['mediagoblin.plugins.floobifier']
+  >>> floobie_dir = floobie_config['floobie_dir']
+  
+A tip: you have access to the `%(here)s` variable in your config,
+which is the directory that the user's mediagoblin config is running
+out of.  So for example, your plugin may need a "floobie" directory to
+store floobs in.  You could give them a reasonable default that makes
+use of the default `user_dev` location, but allow users to override
+it, like so::
+
+  [plugin_spec]
+  floobie_dir = string(default="%(here)s/user_dev/floobs/")
+
+Note, this is relative to the user's mediagoblin config directory,
+*not* your plugin directory!
+
 
 Context Hooks
 -------------
diff --git a/docs/source/pluginwriter/media_type_hooks.rst b/docs/source/pluginwriter/media_type_hooks.rst
new file mode 100644
index 00000000..498b0b54
--- /dev/null
+++ b/docs/source/pluginwriter/media_type_hooks.rst
@@ -0,0 +1,38 @@
+==================
+ Media Type hooks
+==================
+
+This documents the hooks that are currently available for ``media_type`` plugins.
+
+What hooks are available?
+=========================
+
+'sniff_handler'
+---------------
+
+This hook is used by ``sniff_media`` in ``mediagoblin.media_types.__init__``.
+Your media type should return its ``sniff_media`` method when this hook is
+called.
+
+.. Note::
+    Your ``sniff_media`` method should return either the ``media_type`` or
+    ``None``.
+
+'get_media_type_and_manager'
+----------------------------
+
+This hook is used by ``get_media_type_and_manager`` in
+``mediagoblin.media_types.__init__``. When this hook is called, your media type
+plugin should check if it can handle the given extension. If so, your media
+type plugin should return the media type and media manager.
+
+('media_manager', MEDIA_TYPE)
+-----------------------------
+
+If you already know the string representing the media type of a type
+of media, you can pull down the manager specifically.  Note that this
+hook is not a string but a tuple of two strings, the latter being the
+name of the media type.
+
+This is used by media entries to pull down their media managers, and
+so on.
diff --git a/docs/source/siteadmin/deploying.rst b/docs/source/siteadmin/deploying.rst
index 0ee6b5b4..50fc05c5 100644
--- a/docs/source/siteadmin/deploying.rst
+++ b/docs/source/siteadmin/deploying.rst
@@ -157,13 +157,14 @@ directory. Modify these commands to reflect your own environment::
     mkdir -p /srv/mediagoblin.example.org/
     cd /srv/mediagoblin.example.org/
 
-Clone the MediaGoblin repository::
+Clone the MediaGoblin repository and set up the git submodules::
 
     git clone git://gitorious.org/mediagoblin/mediagoblin.git
+    cd mediagoblin
+    git submodule init && git submodule update
 
 And set up the in-package virtualenv::
 
-    cd mediagoblin
     (virtualenv --system-site-packages . || virtualenv .) && ./bin/python setup.py develop
 
 .. note::
@@ -194,7 +195,7 @@ This concludes the initial configuration of the development
 environment. In the future, when you update your
 codebase, you should also run::
 
-    ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate
+    ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate && git submodule fetch
 
 Note: If you are running an active site, depending on your server
 configuration, you may need to stop it first or the dbupdate command
diff --git a/docs/source/siteadmin/media-types.rst b/docs/source/siteadmin/media-types.rst
index 210094b9..3e8a94e9 100644
--- a/docs/source/siteadmin/media-types.rst
+++ b/docs/source/siteadmin/media-types.rst
@@ -18,16 +18,18 @@ Media Types
 ====================
 
 In the future, there will be all sorts of media types you can enable,
-but in the meanwhile there are three additional media types: video, audio
-and ascii art.
+but in the meanwhile there are five additional media types: video, audio,
+ascii art, STL/3d models, PDF and Document.
 
 First, you should probably read ":doc:`configuration`" to make sure
 you know how to modify the mediagoblin config file.
 
-
 Enabling Media Types
 ====================
 
+.. note::
+    Media types are now plugins
+
 Media types are enabled in your mediagoblin configuration file, typically it is
 created by copying ``mediagoblin.ini`` to ``mediagoblin_local.ini`` and then
 applying your changes to ``mediagoblin_local.ini``. If you don't already have a
@@ -37,11 +39,13 @@ Most media types have additional dependencies that you will have to install.
 You will find descriptions on how to satisfy the requirements of each media type
 on this page.
 
-To enable a media type, edit the ``media_types`` list in your
-``mediagoblin_local.ini``. For example, if your system supported image and
-video media types, then the list would look like this::
+To enable a media type, add the the media type under the ``[plugins]`` section
+in you ``mediagoblin_local.ini``. For example, if your system supported image
+and video media types, then it would look like this::
 
-    media_types = mediagoblin.media_types.image, mediagoblin.media_types.video
+    [plugins]
+    [[mediagoblin.media_types.image]]
+    [[mediagoblin.media_types.video]]
 
 Note that after enabling new media types, you must run dbupdate like so::
 
@@ -83,8 +87,8 @@ good/bad/ugly).  On Debianoid systems
         gstreamer0.10-ffmpeg
 
 
-Add ``mediagoblin.media_types.video`` to the ``media_types`` list in your
-``mediagoblin_local.ini`` and restart MediaGoblin.
+Add ``[[mediagoblin.media_types.video]]`` under the ``[plugins]`` section in
+your ``mediagoblin_local.ini`` and restart MediaGoblin.
 
 Run
 
@@ -133,7 +137,7 @@ Then install ``scikits.audiolab`` for the spectrograms::
 
     ./bin/pip install scikits.audiolab
 
-Add ``mediagoblin.media_types.audio`` to the ``media_types`` list in your
+Add ``[[mediagoblin.media_types.audio]]`` under the ``[plugins]`` section in your
 ``mediagoblin_local.ini`` and restart MediaGoblin.
 
 Run
@@ -158,13 +162,8 @@ library, which is necessary for creating thumbnails of ascii art
 
 
 Next, modify (and possibly copy over from ``mediagoblin.ini``) your
-``mediagoblin_local.ini``.  In the ``[mediagoblin]`` section, add
-``mediagoblin.media_types.ascii`` to the ``media_types`` list.
-
-For example, if your system supported image and ascii art media types, then
-the list would look like this::
-
-    media_types = mediagoblin.media_types.image, mediagoblin.media_types.ascii
+``mediagoblin_local.ini``.  In the ``[plugins]`` section, add
+``[[mediagoblin.media_types.ascii]]``.
 
 Run
 
@@ -184,7 +183,7 @@ your execution path.  This feature has been tested with Blender 2.63.
 It may work on some earlier versions, but that is not guaranteed (and
 is surely not to work prior to Blender 2.5X).
 
-Add ``mediagoblin.media_types.stl`` to the ``media_types`` list in your
+Add ``[[mediagoblin.media_types.stl]]`` under the ``[plugins]`` section in your
 ``mediagoblin_local.ini`` and restart MediaGoblin. 
 
 Run
@@ -199,8 +198,16 @@ will be able to present them to your wide audience of admirers!
 PDF and Document
 ================
 
-To enable the "PDF and Document" support plugin, you need pdftocairo, pdfinfo,
-unoconv with headless support.  All executables must be on your execution path.
+To enable the "PDF and Document" support plugin, you need:
+
+1. pdftocairo and pdfinfo for pdf only support.
+
+2. unoconv with headless support to support converting libreoffice supported
+   documents as well, such as doc/ppt/xls/odf/odg/odp and more.
+   For the full list see mediagoblin/media_types/pdf/processing.py,
+   unoconv_supported.
+
+All executables must be on your execution path.
 
 To install this on Fedora:
 
@@ -208,6 +215,9 @@ To install this on Fedora:
 
     sudo yum install -y poppler-utils unoconv libreoffice-headless
 
+Note: You can leave out unoconv and libreoffice-headless if you want only pdf
+support. This will result in a much smaller list of dependencies.
+
 pdf.js relies on git submodules, so be sure you have fetched them:
 
 .. code-block:: bash
@@ -222,7 +232,7 @@ This feature has been tested on Fedora with:
 
 It may work on some earlier versions, but that is not guaranteed.
 
-Add ``mediagoblin.media_types.pdf`` to the ``media_types`` list in your
+Add ``[[mediagoblin.media_types.pdf]]`` under the ``[plugins]`` section in your
 ``mediagoblin_local.ini`` and restart MediaGoblin. 
 
 Run
diff --git a/docs/source/siteadmin/relnotes.rst b/docs/source/siteadmin/relnotes.rst
index 9c9d311c..35415b66 100644
--- a/docs/source/siteadmin/relnotes.rst
+++ b/docs/source/siteadmin/relnotes.rst
@@ -21,11 +21,28 @@ This chapter has important information for releases in it.
 If you're upgrading from a previous release, please read it
 carefully, or at least skim over it.
 
+
+0.4.1
+=====
+
+This is a bugfix release for 0.4.0.  This only implements one major
+fix in the newly released document support which prevented the
+"conversion via libreoffice" feature.
+
+If you were running 0.4.0 you can upgrade to v0.4.1 via a simple
+switch and restarting mediagoblin/celery with no other actions.
+
+Otherwise, follow 0.4.0 instructions.
+
+
 0.4.0
 =====
 
 **Do this to upgrade**
-1. Make sure to run ``bin/gmg dbupdate`` after upgrading.
+
+1. Make sure to run
+   ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate``
+   after upgrading.
 2. See "For Theme authors" if you have a custom theme.
 3. Note that ``./bin/gmg theme assetlink`` is now just
    ``./bin/gmg assetlink`` and covers both plugins and assets.
@@ -39,6 +56,16 @@ carefully, or at least skim over it.
         alias /srv/mediagoblin.example.org/mediagoblin/user_dev/plugin_static/;
      }
 
+   Similarly, if you've got a modified paste config, you may want to
+   borrow the app:plugin_static section from the default paste.ini
+   file.
+5. We now use itsdangerous for sessions; if you had any references to
+   beaker in your paste config you can remove them.  Again, see the
+   default paste.ini config
+6. We also now use git submodules.  Please do:
+   ``git submodule init && git submodule update``
+   You will need to do this to use the new PDF support.
+
 **For theme authors**
 
 If you have your own theme or you have any "user modified templates",
@@ -51,7 +78,23 @@ please note the following:
   You can easily customize this to give a welcome page appropriate to
   your site.
 
+
 **New features**
+* PDF media type!
+* Improved plugin system.  More flexible, better documented, with a
+  new plugin authoring section of the docs.
+* itsdangerous based sessions.  No more beaker!
+* New, experimental Piwigo-based API.  This means you should be able
+  to use MediaGoblin with something like Shotwell.  (Again, a word of
+  caution: this is *very experimental*!)
+* Human readable timestamps, and the option to display the original
+  date of an image when available (available as the
+  "original_date_visible" variable)
+* Moved unit testing system from nosetests to py.test so we can better
+  handle issues with sqlalchemy exploding with different database
+  configurations.  Long story :)
+* You can now disable the ability to post comments.
+* Tags now can be up to length 255 characters by default.
 
 
 0.3.3