diff options
Diffstat (limited to 'docs')
21 files changed, 1258 insertions, 85 deletions
diff --git a/docs/source/api/client_register.rst b/docs/source/api/client_register.rst new file mode 100644 index 00000000..08f92c47 --- /dev/null +++ b/docs/source/api/client_register.rst @@ -0,0 +1,158 @@ +.. MediaGoblin Documentation + + Written in 2011, 2012 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + +==================== +Registering a Client +==================== + +To use the GNU MediaGoblin API you need to use the dynamic client registration. This has been adapted from the `OpenID specification <https://openid.net/specs/openid-connect-registration-1_0.html>`_, this is the only part of OpenID that is being used to serve the purpose to provide the client registration which is used in OAuth. + +The endpoint is ``/api/client/register`` + +The parameters are: + +type + **required** - This must be either *client_associate* (for new registration) or *client_update* + +client_id + **update only** - This should only be used updating client information, this is the client_id given when you register + +client_secret + **update only** - This should only be used updating client information, this is the client_secret given when you register + +contacts + **optional** - This a space seporated list of email addresses to contact of people responsible for the client + +application_type + **required** - This is the type of client you are making, this must be either *web* or *native* + +application_name + **optional** - This is the name of your client + +logo_url + **optional** - This is a URL of the logo image for your client + +redirect_uri + **optional** - This is a space seporated list of pre-registered URLs for use at the Authorization Server + + +Response +-------- + +You will get back a response: + +client_id + This identifies a client + +client_secret + This is the secret. + +expires_at + This is time that the client credentials expire. If this is 0 the client registration does not expire. + +======= +Example +======= + +Register Client +--------------- + +To register a client for the first time, this is the minimum you must supply:: + + { + "type": "client_associate", + "application_type": "native" + } + +A Response will look like:: + + { + "client_secret": "hJtfhaQzgKerlLVdaeRAgmbcstSOBLRfgOinMxBCHcb", + "expires_at": 0, + "client_id": "vwljdhUMhhNbdKizpjZlxv" + } + + +Updating Client +--------------- + +Using the response we got above we can update the information and add new information we may have opted not to supply:: + + { + "type": "client_update", + "client_id": "vwljdhUMhhNbdKizpjZlxv", + "client_secret": "hJtfhaQzgKerlLVdaeRAgmbcstSOBLRfgOinMxBCHcb", + "application_type": "web", + "application_name": "MyClient!", + "logo_url": "https://myclient.org/images/my_logo.png", + "contacts": "myemail@someprovider.com another_developer@provider.net", + } + +The response will just return back the client_id and client_secret you sent:: + + { + "client_id": "vwljdhUMhhNbdKizpjZlxv", + "client_secret": "hJtfhaQzgKerlLVdaeRAgmbcstSOBLRfgOinMxBCHcb", + "expires_at": 0 + } + + +====== +Errors +====== + +There are a number of errors you could get back, This explains what could cause some of them: + +Could not decode data + This is caused when you have an error in the encoding of your data. + +Unknown Content-Type + You should sent a Content-Type header with when you make a request, this should be either application/json or www-form-urlencoded. This is caused when a unknown Content-Type is used. + +No registration type provided + This is when you leave out the ``type``. This should either be client_update or client_associate + +Unknown application_type. + This is when you have provided a ``type`` however this isn't one of the known types. + +client_id is required to update. + When you try and update you need to specify the client_id, this will be what you were given when you initially registered the client. + +client_secret is required to update. + When you try to update you need to specify the client_secrer, this will be what you were given when you initially register the client. + +Unauthorized. + This is when you are trying to update however the client_id and/or client_secret you have submitted are incorrect. + +Only set client_id for update. + This should only be given when you update. + +Only set client_secret for update. + This should only be given when you update. + +Logo URL <url> is not a valid URL + This is when the URL specified did not meet the validation. + +contacts must be a string of space-separated email addresses. + ``contacts`` should be a string (not a list), ensure each email is seporated by a space + +Email <email> is not a valid email + This is when you have submitted an invalid email address + +redirect_uris must be space-separated URLs. + ``redirect_uris`` should be a string (not a list), ensure each URL is seporated by a space + +URI <URI> is not a valid URI + This is when your URI is invalid. + + diff --git a/docs/source/api/oauth.rst b/docs/source/api/oauth.rst new file mode 100644 index 00000000..003ad492 --- /dev/null +++ b/docs/source/api/oauth.rst @@ -0,0 +1,36 @@ +.. MediaGoblin Documentation + + Written in 2011, 2012 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + +============== +Authentication +============== + +GNU MediaGoblin uses OAuth1 to authenticate requests to the API. There are many +libraries out there for OAuth1, you're likely not going to have to do much. There +is a library for the GNU MediaGoblin called `PyPump <https://github.com/xray7224/PyPump>`_. +We are not using OAuth2 as we want to stay completely compatable with GNU MediaGoblin. + + +We use :doc:`client_register` to get the client ID and secret. + +Endpoints +--------- + +These are the endpoints you need to use for the oauth requests: + +`/oauth/request_token` is for getting the request token. + +`/oauth/authorize` is to send the user to to authorize your application. + +`/oauth/access_token` is for getting the access token to use in requests. + diff --git a/docs/source/devel/migrations.rst b/docs/source/devel/migrations.rst new file mode 100644 index 00000000..16c02b04 --- /dev/null +++ b/docs/source/devel/migrations.rst @@ -0,0 +1,62 @@ +.. MediaGoblin Documentation + + Written in 2011, 2012 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + +========== +Migrations +========== + +So, about migrations. Every time we change the way the database +structure works, we need to add a migration so that people running +older codebases can have their databases updated to the new structure +when they run `./bin/gmg dbupdate`. + +The first time `./bin/gmg dbupdate` is run by a user, it creates the +tables at the current state that they're defined in models.py and sets +the migration number to the current migration... after all, migrations +only exist to get things to the current state of the db. After that, +every migration is run with dbupdate. + +There's a few things you need to know: + +- We use `sqlalchemy-migrate + <http://code.google.com/p/sqlalchemy-migrate/>`_. + See `their docs <https://sqlalchemy-migrate.readthedocs.org/>`_. +- `Alembic <https://bitbucket.org/zzzeek/alembic>`_ might be a better + choice than sqlalchemy-migrate now or in the future, but we + originally decided not to use it because it didn't have sqlite + support. It's not clear if that's changed. +- SQLAlchemy has two parts to it, the ORM and the "core" interface. + We DO NOT use the ORM when running migrations. Think about it: the + ORM is set up with an expectation that the models already reflect a + certain pattern. But if a person is moving from their old patern + and are running tools to *get to* the current pattern, of course + their current database structure doesn't match the state of the ORM! +- How to write migrations? Maybe there will be a tutorial here in the + future... in the meanwhile, look at existing migrations in + `mediagoblin/db/migrations.py` and look in + `mediagoblin/tests/test_sql_migrations.py` for examples. +- Common pattern: use `inspect_table` to get the current state + of the table before we run alterations on it. +- Make sure you set the RegisterMigration to be the next migration in + order. +- What happens if you're adding a *totally new* table? In this case, + you should copy the table in entirety as it exists into + migrations.py then create the tables based off of that... see + add_collection_tables. This is easier than reproducing the SQL by + hand. +- If you're writing a feature branch, you don't need to keep adding + migrations every time you change things around if your database + structure is in flux. Just alter your migrations so that they're + correct for the merge into master. + +That's it for now! Good luck! diff --git a/docs/source/index.rst b/docs/source/index.rst index 7f692d57..3ead6136 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -44,6 +44,7 @@ MediaGoblin website. It is written for site administrators. siteadmin/relnotes siteadmin/theming siteadmin/plugins + siteadmin/commandline-upload .. _core-plugin-section: @@ -56,9 +57,12 @@ Part 2: Core plugin documentation plugindocs/flatpagesfile plugindocs/sampleplugin - plugindocs/oauth plugindocs/trim_whitespace plugindocs/raven + plugindocs/basic_auth + plugindocs/openid + plugindocs/persona + plugindocs/ldap Part 3: Plugin Writer's Guide @@ -73,6 +77,9 @@ This guide covers writing new GNU MediaGoblin plugins. pluginwriter/quickstart pluginwriter/database pluginwriter/api + pluginwriter/tests + pluginwriter/media_type_hooks + pluginwriter/authhooks Part 4: Developer's Zone @@ -86,6 +93,7 @@ This chapter contains various information for developers. devel/codebase devel/storage devel/originaldesigndecisions + devel/migrations Indices and tables diff --git a/docs/source/plugindocs/basic_auth.rst b/docs/source/plugindocs/basic_auth.rst new file mode 100644 index 00000000..83492ac2 --- /dev/null +++ b/docs/source/plugindocs/basic_auth.rst @@ -0,0 +1,2 @@ +.. include:: ../../../mediagoblin/plugins/basic_auth/README.rst + diff --git a/docs/source/plugindocs/ldap.rst b/docs/source/plugindocs/ldap.rst new file mode 100644 index 00000000..3938c0c7 --- /dev/null +++ b/docs/source/plugindocs/ldap.rst @@ -0,0 +1,2 @@ +.. include:: ../../../mediagoblin/plugins/ldap/README.rst + diff --git a/docs/source/plugindocs/openid.rst b/docs/source/plugindocs/openid.rst new file mode 100644 index 00000000..045bf9d0 --- /dev/null +++ b/docs/source/plugindocs/openid.rst @@ -0,0 +1,2 @@ +.. include:: ../../../mediagoblin/plugins/openid/README.rst + diff --git a/docs/source/plugindocs/persona.rst b/docs/source/plugindocs/persona.rst new file mode 100644 index 00000000..2524127d --- /dev/null +++ b/docs/source/plugindocs/persona.rst @@ -0,0 +1,2 @@ +.. include:: ../../../mediagoblin/plugins/persona/README.rst + diff --git a/docs/source/plugindocs/raven.rst b/docs/source/plugindocs/raven.rst index 71e284d0..ae96f3f8 100644 --- a/docs/source/plugindocs/raven.rst +++ b/docs/source/plugindocs/raven.rst @@ -1,2 +1 @@ -.. _raven-setup: Set up the raven plugin .. include:: ../../../mediagoblin/plugins/raven/README.rst diff --git a/docs/source/pluginwriter/api.rst b/docs/source/pluginwriter/api.rst index 3a75d455..29adb691 100644 --- a/docs/source/pluginwriter/api.rst +++ b/docs/source/pluginwriter/api.rst @@ -11,6 +11,7 @@ Dedication along with this software. If not, see <http://creativecommons.org/publicdomain/zero/1.0/>. +.. _plugin-api-chapter: ========== Plugin API @@ -23,7 +24,27 @@ Authors are encouraged to develop plugins and work with the MediaGoblin community to keep them up to date, but this API will be a moving target for a few releases. -Please check the release notes for updates! +Please check the :ref:`release-notes` for updates! + + +How are hooks added? Where do I find them? +------------------------------------------- + +Much of this document talks about hooks, both as in terms of regular +hooks and template hooks. But where do they come from, and how can +you find a list of them? + +For the moment, the best way to find available hooks is to check the +source code itself. (Yes, we should start a more official hook +listing with descriptions soon.) But many hooks you may need do not +exist yet: what to do then? + +The plan at present is that we are adding hooks as people need them, +with community discussion. If you find that you need a hook and +MediaGoblin at present doesn't provide it at present, please +`http://mediagoblin.org/pages/join.html <talk to us>`_! We'll +evaluate what to do from there. + :mod:`pluginapi` Module ----------------------- @@ -31,4 +52,271 @@ Please check the release notes for updates! .. automodule:: mediagoblin.tools.pluginapi :members: get_config, register_routes, register_template_path, register_template_hooks, get_hook_templates, - hook_handle, hook_runall, hook_transform, + hook_handle, hook_runall, hook_transform + +Configuration +------------- + +Your plugin may define its own configuration defaults. + +Simply add to the directory of your plugin a config_spec.ini file. An +example might look like:: + + [plugin_spec] + some_string = string(default="blork") + some_int = integer(default=50) + +This means that when people enable your plugin in their config you'll +be able to provide defaults as well as type validation. + +You can access this via the app_config variables in mg_globals, or you +can use a shortcut to get your plugin's config section:: + + >>> from mediagoblin.tools import pluginapi + # Replace with the path to your plugin. + # (If an external package, it won't be part of mediagoblin.plugins) + >>> floobie_config = pluginapi.get_config('mediagoblin.plugins.floobifier') + >>> floobie_dir = floobie_config['floobie_dir'] + # This is the same as the above + >>> from mediagoblin import mg_globals + >>> config = mg_globals.global_config['plugins']['mediagoblin.plugins.floobifier'] + >>> floobie_dir = floobie_config['floobie_dir'] + +A tip: you have access to the `%(here)s` variable in your config, +which is the directory that the user's mediagoblin config is running +out of. So for example, your plugin may need a "floobie" directory to +store floobs in. You could give them a reasonable default that makes +use of the default `user_dev` location, but allow users to override +it, like so:: + + [plugin_spec] + floobie_dir = string(default="%(here)s/user_dev/floobs/") + +Note, this is relative to the user's mediagoblin config directory, +*not* your plugin directory! + + +Context Hooks +------------- + +View specific hooks ++++++++++++++++++++ + +You can hook up to almost any template called by any specific view +fairly easily. As long as the view directly or indirectly uses the +method ``render_to_response`` you can access the context via a hook +that has a key in the format of the tuple:: + + (view_symbolic_name, view_template_path) + +Where the "view symbolic name" is the same parameter used in +``request.urlgen()`` to look up the view. So say we're wanting to add +something to the context of the user's homepage. We look in +mediagoblin/user_pages/routing.py and see:: + + add_route('mediagoblin.user_pages.user_home', + '/u/<string:user>/', + 'mediagoblin.user_pages.views:user_home') + +Aha! That means that the name is ``mediagoblin.user_pages.user_home``. +Okay, so then we look at the view at the +``mediagoblin.user_pages.user_home`` method:: + + @uses_pagination + def user_home(request, page): + # [...] whole bunch of stuff here + return render_to_response( + request, + 'mediagoblin/user_pages/user.html', + {'user': user, + 'user_gallery_url': user_gallery_url, + 'media_entries': media_entries, + 'pagination': pagination}) + +Nice! So the template appears to be +``mediagoblin/user_pages/user.html``. Cool, that means that the key +is:: + + ("mediagoblin.user_pages.user_home", + "mediagoblin/user_pages/user.html") + +The context hook uses ``hook_transform()`` so that means that if we're +hooking into it, our hook will both accept one argument, ``context``, +and should return that modified object, like so:: + + def add_to_user_home_context(context): + context['foo'] = 'bar' + return context + + hooks = { + ("mediagoblin.user_pages.user_home", + "mediagoblin/user_pages/user.html"): add_to_user_home_context} + + +Global context hooks +++++++++++++++++++++ + +If you need to add something to the context of *every* view, it is not +hard; there are two hooks hook that also uses hook_transform (like the +above) but make available what you are providing to *every* view. + +Note that there is a slight, but critical, difference between the two. + +The most general one is the ``'template_global_context'`` hook. This +one is run only once, and is read into the global context... all views +will get access to what are in this dict. + +The slightly more expensive but more powerful one is +``'template_context_prerender'``. This one is not added to the global +context... it is added to the actual context of each individual +template render right before it is run! Because of this you also can +do some powerful and crazy things, such as checking the request object +or other parts of the context before passing them on. + + +Adding static resources +----------------------- + +It's possible to add static resources for your plugin. Say your +plugin needs some special javascript and images... how to provide +them? Then how to access them? MediaGoblin has a way! + + +Attaching to the hook ++++++++++++++++++++++ + +First, you need to register your plugin's resources with the hook. +This is pretty easy actually: you just need to provide a function that +passes back a PluginStatic object. + +.. autoclass:: mediagoblin.tools.staticdirect.PluginStatic + + +Running plugin assetlink +++++++++++++++++++++++++ + +In order for your plugin assets to be properly served by MediaGoblin, +your plugin's asset directory needs to be symlinked into the directory +that plugin assets are served from. To set this up, run:: + + ./bin/gmg assetlink + + +Using staticdirect +++++++++++++++++++ + +Once you have this, you will want to be able to of course link to your +assets! MediaGoblin has a "staticdirect" tool; you want to use this +like so in your templates:: + + staticdirect("css/monkeys.css", "mystaticname") + +Replace "mystaticname" with the name you passed to PluginStatic. The +staticdirect method is, for convenience, attached to the request +object, so you can access this in your templates like: + +.. code-block:: html + + <img alt="A funny bunny" + src="{{ request.staticdirect('images/funnybunny.png', 'mystaticname') }}" /> + + +Additional hook tips +-------------------- + +This section aims to explain some tips in regards to adding hooks to +the MediaGoblin repository. + +WTForms hooks ++++++++++++++ + +We haven't totally settled on a way to tranform wtforms form objects, +but here's one way. In your view:: + + from mediagoblin.foo.forms import SomeForm + + def some_view(request) + form_class = hook_transform('some_form_transform', SomeForm) + form = form_class(request.form) + +Then to hook into this form, do something in your plugin like:: + + import wtforms + + class SomeFormAdditions(wtforms.Form): + new_datefield = wtforms.DateField() + + def transform_some_form(orig_form): + class ModifiedForm(orig_form, SomeFormAdditions) + return ModifiedForm + + hooks = { + 'some_form_transform': transform_some_form} + + +Interfaces +++++++++++ + +If you want to add a pseudo-interface, it's not difficult to do so. +Just write the interface like so:: + + class FrobInterface(object): + """ + Interface for Frobbing. + + Classes implementing this interface should provide defrob and frob. + They may also implement double_frob, but it is not required; if + not provided, we will use a general technique. + """ + + def defrob(self, frobbed_obj): + """ + Take a frobbed_obj and defrob it. Returns the defrobbed object. + """ + raise NotImplementedError() + + def frob(self, normal_obj): + """ + Take a normal object and frob it. Returns the frobbed object. + """ + raise NotImplementedError() + + def double_frob(self, normal_obj): + """ + Frob this object and return it multiplied by two. + """ + return self.frob(normal_obj) * 2 + + + def some_frob_using_method(): + # something something something + frobber = hook_handle(FrobInterface) + frobber.frob(blah) + + # alternately you could have a default + frobber = hook_handle(FrobInterface) or DefaultFrobber + frobber.defrob(foo) + + +It's fine to use your interface as the key instead of a string if you +like. (Usually this is messy, but since interfaces are public and +since you need to import them into your plugin anyway, interfaces +might as well be keys.) + +Then a plugin providing your interface can be like:: + + from mediagoblin.foo.frobfrogs import FrobInterface + from frogfrobber import utils + + class FrogFrobber(FrobInterface): + """ + Takes a frogputer science approach to frobbing. + """ + def defrob(self, frobbed_obj): + return utils.frog_defrob(frobbed_obj) + + def frob(self, normal_obj): + return utils.frog_frob(normal_obj) + + hooks = { + FrobInterface: lambda: return FrogFrobber} diff --git a/docs/source/pluginwriter/authhooks.rst b/docs/source/pluginwriter/authhooks.rst new file mode 100644 index 00000000..9721d729 --- /dev/null +++ b/docs/source/pluginwriter/authhooks.rst @@ -0,0 +1,86 @@ +====================== + Authentication Hooks +====================== + +This documents the hooks that are currently available for authentication +plugins. If you need new hooks for your plugin, go ahead a submit a patch. + +What hooks are available? +========================= + +'authentication' +---------------- + +This hook just needs to return ``True`` as this is how +the MediaGoblin app knows that an authentication plugin is enabled. + + +'auth_extra_validation' +----------------------- + +This hook is used to provide any additional validation of the registration +form when using ``mediagoblin.auth.tools.register_user()``. This hook runs +through all enabled auth plugins. + + +'auth_create_user' +------------------ + +This hook is used by ``mediagoblin.auth.tools.register_user()`` so plugins can +store the necessary information when creating a user. This hook runs through +all enabled auth plugins. + +'auth_get_user' +--------------- + +This hook is used by ``mediagoblin.auth.tools.check_login_simple()``. Your +plugin should return a ``User`` object given a username. + +'auth_no_pass_redirect' +----------------------- + +This hook is called in ``mediagoblin.auth.views`` in both the ``login`` and +``register`` views. This hook should return the name of your plugin, so that +if :ref:`basic_auth-chapter` is not enabled, the user will be redirected to the +correct login and registration views for your plugin. + +The code assumes that it can generate a valid url given +``mediagoblin.plugins.{{ your_plugin_here }}.login`` and +``mediagoblin.plugins.{{ your_plugin_here }}.register``. This is only needed if +you will not be using the ``login`` and ``register`` views in +``mediagoblin.auth.views``. + +'auth_get_login_form' +--------------------- + +This hook is called in ``mediagoblin.auth.views.login()``. If you are not using +that view, then you do not need this hook. This hook should take a ``request`` +object and return the ``LoginForm`` for your plugin. + +'auth_get_registration_form' +---------------------------- + +This hook is called in ``mediagoblin.auth.views.register()``. If you are not +using that view, then you do not need this hook. This hook should take a +``request`` object and return the ``RegisterForm`` for your plugin. + +'auth_gen_password_hash' +------------------------ + +This hook should accept a ``raw_pass`` and an ``extra_salt`` and return a +hashed password to be stored in ``User.pw_hash``. + +'auth_check_password' +--------------------- + +This hook should accept a ``raw_pass``, a ``stored_hash``, and an ``extra_salt``. +Your plugin should then check that the ``raw_pass`` hashes to the same thing as +the ``stored_hash`` and return either ``True`` or ``False``. + +'auth_fake_login_attempt' +------------------------- + +This hook is called in ``mediagoblin.auth.tools.check_login_simple``. It is +called if a user is not found and should do something that takes the same amount +of time as your ``check_password`` function. This is to help prevent timining +attacks. diff --git a/docs/source/pluginwriter/database.rst b/docs/source/pluginwriter/database.rst index 58edf3a0..603a19eb 100644 --- a/docs/source/pluginwriter/database.rst +++ b/docs/source/pluginwriter/database.rst @@ -12,9 +12,12 @@ <http://creativecommons.org/publicdomain/zero/1.0/>. -======== -Database -======== +.. _plugin-database-chapter: + + +=========================== +Database models for plugins +=========================== Accessing Existing Data diff --git a/docs/source/pluginwriter/media_type_hooks.rst b/docs/source/pluginwriter/media_type_hooks.rst new file mode 100644 index 00000000..498b0b54 --- /dev/null +++ b/docs/source/pluginwriter/media_type_hooks.rst @@ -0,0 +1,38 @@ +================== + Media Type hooks +================== + +This documents the hooks that are currently available for ``media_type`` plugins. + +What hooks are available? +========================= + +'sniff_handler' +--------------- + +This hook is used by ``sniff_media`` in ``mediagoblin.media_types.__init__``. +Your media type should return its ``sniff_media`` method when this hook is +called. + +.. Note:: + Your ``sniff_media`` method should return either the ``media_type`` or + ``None``. + +'get_media_type_and_manager' +---------------------------- + +This hook is used by ``get_media_type_and_manager`` in +``mediagoblin.media_types.__init__``. When this hook is called, your media type +plugin should check if it can handle the given extension. If so, your media +type plugin should return the media type and media manager. + +('media_manager', MEDIA_TYPE) +----------------------------- + +If you already know the string representing the media type of a type +of media, you can pull down the manager specifically. Note that this +hook is not a string but a tuple of two strings, the latter being the +name of the media type. + +This is used by media entries to pull down their media managers, and +so on. diff --git a/docs/source/pluginwriter/quickstart.rst b/docs/source/pluginwriter/quickstart.rst index b5a63f79..6d45ea36 100644 --- a/docs/source/pluginwriter/quickstart.rst +++ b/docs/source/pluginwriter/quickstart.rst @@ -178,8 +178,10 @@ That's it for the quick start! Where to go from here ===================== -See the documentation on the plugin API for code samples and other -things you can use when building your plugin. +See the documentation on the :ref:`plugin-api-chapter` for code +samples and other things you can use when building your plugin. If +your plugin needs its own database models, see +:ref:`plugin-database-chapter`. See `Hitchhiker's Guide to Packaging <http://guide.python-distribute.org/>`_ for more information on diff --git a/docs/source/pluginwriter/tests.rst b/docs/source/pluginwriter/tests.rst new file mode 100644 index 00000000..fe99688f --- /dev/null +++ b/docs/source/pluginwriter/tests.rst @@ -0,0 +1,64 @@ +.. MediaGoblin Documentation + + Written in 2013 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + +============================== +Writing unit tests for plugins +============================== + +Here's a brief guide to writing unit tests for plugins. However, it +isn't really ideal. It also hasn't been well tested... yes, there's +some irony there :) + +Some notes: we're using py.test and webtest for unit testing stuff. +Keep that in mind. + +My suggestion is to mime the behavior of `mediagoblin/tests/` and put +that in your own plugin, like `myplugin/tests/`. Copy over +`conftest.py` and `pytest.ini` to your tests directory, but possibly +change the `test_app` fixture to match your own tests' config needs. +For example:: + + import pkg_resources + # [...] + + @pytest.fixture() + def test_app(request): + return get_app( + request, + mgoblin_config=pkg_resources.resource_filename( + 'myplugin.tests', 'myplugin_mediagoblin.ini')) + +In any test module in your tests directory you can then do:: + + def test_somethingorother(test_app): + # real code goes here + pass + +And you'll get a mediagoblin application wrapped in webtest passed in +to your environment. + +If your plugin needs to define multiple configuration setups, you can +actually set up multiple fixtures very easily for this. You can just +set up multiple fixtures with different names that point to different +configs and pass them in as that named argument. + +To run the tests, from mediagoblin's directory (make sure that your +plugin has been added to your mediagoblin checkout's virtualenv!) do:: + + ./runtests.sh /path/to/myplugin/tests/ + +replacing `/path/to/myplugin/` with the actual path to your plugin. + +NOTE: again, the above is untested, but it should probably work. If +you run into trouble, `contact us +<http://mediagoblin.org/pages/join.html>`_, preferably on IRC! diff --git a/docs/source/siteadmin/commandline-upload.rst b/docs/source/siteadmin/commandline-upload.rst new file mode 100644 index 00000000..be19df58 --- /dev/null +++ b/docs/source/siteadmin/commandline-upload.rst @@ -0,0 +1,41 @@ +.. MediaGoblin Documentation + + Written in 2011, 2012 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + +====================== +Command-line uploading +====================== + +Want to submit media via the command line? It's fairly easy to do:: + + ./bin/gmg addmedia username your_media.jpg + +This will submit the file "your_media.jpg" to be a media entry +associated with the user "username". + +You can get help on all the available options by running:: + + ./bin/gmg addmedia --help + +Here's a longer example that makes use of more options:: + + ./bin/gmg addmedia aveyah awesome_spaceship.png \ + --title "My awesome spaceship" \ + --description "Flying my awesome spaceship, since I'm an awesome pilot" \ + --license "http://creativecommons.org/licenses/by-sa/3.0/" \ + --tags "spaceships, pilots, awesome" \ + --slug "awesome-spaceship" + +You can also pass in the `--celery` option if you would prefer that +your media be passed over to celery to be processed rather than be +processed immediately. + diff --git a/docs/source/siteadmin/deploying.rst b/docs/source/siteadmin/deploying.rst index 77e60037..de4ce1ac 100644 --- a/docs/source/siteadmin/deploying.rst +++ b/docs/source/siteadmin/deploying.rst @@ -1,6 +1,6 @@ .. MediaGoblin Documentation - Written in 2011, 2012 by MediaGoblin contributors + Written in 2011, 2012, 2013 by MediaGoblin contributors To the extent possible under law, the author(s) have dedicated all copyright and related and neighboring rights to this software to @@ -77,7 +77,7 @@ Configure PostgreSQL If you don't want/need postgres, skip this section. -These are the packages needed for Debian Wheezy (testing):: +These are the packages needed for Debian Wheezy (stable):: sudo apt-get install postgresql postgresql-client python-psycopg2 @@ -121,25 +121,62 @@ where the first ``mediagoblin`` is the database owner and the second Drop Privileges for MediaGoblin ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -As MediaGoblin does not require special permissions or elevated -access, you should run MediaGoblin under an existing non-root user or -preferably create a dedicated user for the purpose of running -MediaGoblin. Consult your distribution's documentation on how to -create "system account" or dedicated service user. Ensure that it is -not possible to log in to your system with as this user. +MediaGoblin does not require special permissions or elevated +access to run. As such, the prefered way to run MediaGoblin is to +create a dedicated, unpriviledged system user for sole the purpose of running +MediaGoblin. Running MediaGoblin processes under an unpriviledged system user +helps to keep it more secure. + +The following command (entered as root or with sudo) will create a +system account with a username of ``mediagoblin``. You may choose a different +username if you wish.:: + + adduser --system mediagoblin + +No password will be assigned to this account, and you will not be able +to log in as this user. To switch to this account, enter either:: + + sudo su - mediagoblin (if you have sudo permissions) + +or:: + + su - mediagoblin (if you have to use root permissions) + +You may get a warning similar to this when entering these commands:: + + warning: cannot change directory to /home/mediagoblin: No such file or directory + +You can disregard this warning. To return to your regular user account after +using the system account, just enter ``exit``. + +.. note:: + + Unless otherwise noted, the remainder of this document assumes that all + operations are performed using this unpriviledged account. + +.. _create-mediagoblin-directory: + +Create a MediaGoblin Directory +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You should create a working directory for MediaGoblin. This document assumes your local git repository will be located at -``/srv/mediagoblin.example.org/mediagoblin/`` for this documentation. -Substitute your prefer ed local deployment path as needed. +``/srv/mediagoblin.example.org/mediagoblin/``. +Substitute your prefered local deployment path as needed. + +Setting up the working directory requires that we first create the directory +with elevated priviledges, and then assign ownership of the directory +to the unpriviledged system account. + +To do this, enter either of the following commands, changing the defaults +to suit your particular requirements:: -This document assumes that all operations are performed as this -user. To drop privileges to this user, run the following command:: + sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:mediagoblin /srv/mediagobin.example.org - su - [mediagoblin] +or (as the root user):: + + mkdir -p /srv/mediagoblin.example.org && chown -hR mediagoblin:mediagoblin /srv/mediagobin.example.org -Where, "``[mediagoblin]``" is the username of the system user that will -run MediaGoblin. Install MediaGoblin and Virtualenv ---------------------------------- @@ -151,23 +188,40 @@ Install MediaGoblin and Virtualenv branch of the git repository. Eventually production deployments will want to transition to running from more consistent releases. -Issue the following commands, to create and change the working -directory. Modify these commands to reflect your own environment:: +We will now clone the MediaGoblin source code repository and setup and +configure the necessary services. Modify these commands to +suit your own environment. As a reminder, you should enter these +commands using your unpriviledged system account. + +Change to the MediaGoblin directory that you just created:: - mkdir -p /srv/mediagoblin.example.org/ - cd /srv/mediagoblin.example.org/ + cd /srv/mediagoblin.example.org -Clone the MediaGoblin repository:: +Clone the MediaGoblin repository and set up the git submodules:: git clone git://gitorious.org/mediagoblin/mediagoblin.git + cd mediagoblin + git submodule init && git submodule update + And set up the in-package virtualenv:: - cd mediagoblin (virtualenv --system-site-packages . || virtualenv .) && ./bin/python setup.py develop .. note:: + We presently have an experimental make-style deployment system. if + you'd like to try it, instead of the above command, you can run:: + + ./bootstrap.sh && ./configure && make + + This also includes a number of nice features, such as keeping your + viratualenv up to date by simply running `make update`. + +.. :: + + (NOTE: Is this still relevant?) + If you have problems here, consider trying to install virtualenv with the ``--distribute`` or ``--no-site-packages`` options. If your system's default Python is in the 3.x series you may need to @@ -194,7 +248,7 @@ This concludes the initial configuration of the development environment. In the future, when you update your codebase, you should also run:: - ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate + ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate && git submodule fetch Note: If you are running an active site, depending on your server configuration, you may need to stop it first or the dbupdate command @@ -205,6 +259,23 @@ update) Deploy MediaGoblin Services --------------------------- +Edit site configuration +~~~~~~~~~~~~~~~~~~~~~~~ + +A few basic properties must be set before MediaGoblin will work. First +make a copy of ``mediagoblin.ini`` for editing so the original config +file isn't lost:: + + cp mediagoblin.ini mediagoblin_local.ini + +Then: + - Set ``email_sender_address`` to the address you wish to be used as + the sender for system-generated emails + - Edit ``direct_remote_path``, ``base_dir``, and ``base_url`` if + your mediagoblin directory is not the root directory of your + vhost. + + Configure MediaGoblin to use the PostgreSQL database ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -240,11 +311,11 @@ browser to confirm that the service is operable. .. _webserver-config: -Connect the Webserver to MediaGoblin with FastCGI -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This section describes how to configure MediaGoblin to work via -FastCGI. Our configuration example will use nginx, however, you may +FastCGI and nginx +~~~~~~~~~~~~~~~~~ + +This configuration example will use nginx, however, you may use any webserver of your choice as long as it supports the FastCGI protocol. If you do not already have a web server, consider nginx, as the configuration files may be more clear than the @@ -310,6 +381,11 @@ this ``nginx.conf`` file should be modeled on the following:: alias /srv/mediagoblin.example.org/mediagoblin/user_dev/theme_static/; } + # Plugin static files (usually symlinked in) + location /plugin_static/ { + alias /srv/mediagoblin.example.org/mediagoblin/user_dev/plugin_static/; + } + # Mounting MediaGoblin itself via FastCGI. location / { fastcgi_pass 127.0.0.1:26543; @@ -345,3 +421,25 @@ Visit the site you've set up in your browser by visiting smaller deployments. However, for larger production deployments with larger processing requirements, see the ":doc:`production-deployments`" documentation. + + +Apache +~~~~~~ + +Instructions and scripts for running MediaGoblin on an Apache server +can be found on the `MediaGoblin wiki <http://wiki.mediagoblin.org/Deployment>`_. + + +Security Considerations +~~~~~~~~~~~~~~~~~~~~~~~ + +.. warning:: + + The directory ``user_dev/crypto/`` contains some very + sensitive files. + Especially the ``itsdangeroussecret.bin`` is very important + for session security. Make sure not to leak its contents anywhere. + If the contents gets leaked nevertheless, delete your file + and restart the server, so that it creates a new secret key. + All previous sessions will be invalidated. + diff --git a/docs/source/siteadmin/media-types.rst b/docs/source/siteadmin/media-types.rst index 210094b9..3e8a94e9 100644 --- a/docs/source/siteadmin/media-types.rst +++ b/docs/source/siteadmin/media-types.rst @@ -18,16 +18,18 @@ Media Types ==================== In the future, there will be all sorts of media types you can enable, -but in the meanwhile there are three additional media types: video, audio -and ascii art. +but in the meanwhile there are five additional media types: video, audio, +ascii art, STL/3d models, PDF and Document. First, you should probably read ":doc:`configuration`" to make sure you know how to modify the mediagoblin config file. - Enabling Media Types ==================== +.. note:: + Media types are now plugins + Media types are enabled in your mediagoblin configuration file, typically it is created by copying ``mediagoblin.ini`` to ``mediagoblin_local.ini`` and then applying your changes to ``mediagoblin_local.ini``. If you don't already have a @@ -37,11 +39,13 @@ Most media types have additional dependencies that you will have to install. You will find descriptions on how to satisfy the requirements of each media type on this page. -To enable a media type, edit the ``media_types`` list in your -``mediagoblin_local.ini``. For example, if your system supported image and -video media types, then the list would look like this:: +To enable a media type, add the the media type under the ``[plugins]`` section +in you ``mediagoblin_local.ini``. For example, if your system supported image +and video media types, then it would look like this:: - media_types = mediagoblin.media_types.image, mediagoblin.media_types.video + [plugins] + [[mediagoblin.media_types.image]] + [[mediagoblin.media_types.video]] Note that after enabling new media types, you must run dbupdate like so:: @@ -83,8 +87,8 @@ good/bad/ugly). On Debianoid systems gstreamer0.10-ffmpeg -Add ``mediagoblin.media_types.video`` to the ``media_types`` list in your -``mediagoblin_local.ini`` and restart MediaGoblin. +Add ``[[mediagoblin.media_types.video]]`` under the ``[plugins]`` section in +your ``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -133,7 +137,7 @@ Then install ``scikits.audiolab`` for the spectrograms:: ./bin/pip install scikits.audiolab -Add ``mediagoblin.media_types.audio`` to the ``media_types`` list in your +Add ``[[mediagoblin.media_types.audio]]`` under the ``[plugins]`` section in your ``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -158,13 +162,8 @@ library, which is necessary for creating thumbnails of ascii art Next, modify (and possibly copy over from ``mediagoblin.ini``) your -``mediagoblin_local.ini``. In the ``[mediagoblin]`` section, add -``mediagoblin.media_types.ascii`` to the ``media_types`` list. - -For example, if your system supported image and ascii art media types, then -the list would look like this:: - - media_types = mediagoblin.media_types.image, mediagoblin.media_types.ascii +``mediagoblin_local.ini``. In the ``[plugins]`` section, add +``[[mediagoblin.media_types.ascii]]``. Run @@ -184,7 +183,7 @@ your execution path. This feature has been tested with Blender 2.63. It may work on some earlier versions, but that is not guaranteed (and is surely not to work prior to Blender 2.5X). -Add ``mediagoblin.media_types.stl`` to the ``media_types`` list in your +Add ``[[mediagoblin.media_types.stl]]`` under the ``[plugins]`` section in your ``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -199,8 +198,16 @@ will be able to present them to your wide audience of admirers! PDF and Document ================ -To enable the "PDF and Document" support plugin, you need pdftocairo, pdfinfo, -unoconv with headless support. All executables must be on your execution path. +To enable the "PDF and Document" support plugin, you need: + +1. pdftocairo and pdfinfo for pdf only support. + +2. unoconv with headless support to support converting libreoffice supported + documents as well, such as doc/ppt/xls/odf/odg/odp and more. + For the full list see mediagoblin/media_types/pdf/processing.py, + unoconv_supported. + +All executables must be on your execution path. To install this on Fedora: @@ -208,6 +215,9 @@ To install this on Fedora: sudo yum install -y poppler-utils unoconv libreoffice-headless +Note: You can leave out unoconv and libreoffice-headless if you want only pdf +support. This will result in a much smaller list of dependencies. + pdf.js relies on git submodules, so be sure you have fetched them: .. code-block:: bash @@ -222,7 +232,7 @@ This feature has been tested on Fedora with: It may work on some earlier versions, but that is not guaranteed. -Add ``mediagoblin.media_types.pdf`` to the ``media_types`` list in your +Add ``[[mediagoblin.media_types.pdf]]`` under the ``[plugins]`` section in your ``mediagoblin_local.ini`` and restart MediaGoblin. Run diff --git a/docs/source/siteadmin/production-deployments.rst b/docs/source/siteadmin/production-deployments.rst index 1a32d95e..839d3ce5 100644 --- a/docs/source/siteadmin/production-deployments.rst +++ b/docs/source/siteadmin/production-deployments.rst @@ -22,12 +22,14 @@ MediaGoblin in actual production environments. Consider Deploy with Paste ----------------- -The instance configured with ``./lazyserver.sh`` is not ideal for a -production MediaGoblin deployment. Ideally, you should be able to use -an "init" or "control" script to launch and restart the MediaGoblin +The MediaGoblin WSGI application instance you get with ``./lazyserver.sh`` is +not ideal for a production MediaGoblin deployment. Ideally, you should be able +to use an "init" or "control" script to launch and restart the MediaGoblin process. -Use the following command as the basis for such a script: :: +Use the following command as the basis for such a script: + +.. code-block:: bash CELERY_ALWAYS_EAGER=true \ /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ @@ -41,7 +43,9 @@ synchronously, and the user will advance to the next page only after processing is complete. If we take Celery out of "always eager mode," the user will be able to immediately return to the MediaGoblin site while processing is ongoing. In these cases, use the following command -as the basis for your script: :: +as the basis for your script: + +.. code-block:: bash CELERY_ALWAYS_EAGER=false \ /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ @@ -52,30 +56,40 @@ as the basis for your script: :: Separate Celery --------------- -While the ``./lazyserver.sh`` configuration provides an efficient way to -start using a MediaGoblin instance, it is not suitable for production -deployments for several reasons: +MediaGoblin uses `Celery`_ to handle heavy and long-running tasks. Celery can +be launched in two ways: + +1. Embedded in the MediaGoblin WSGI application [#f-mediagoblin-wsgi-app]_. + This is the way ``./lazyserver.sh`` does it for you. It's simple as you + only have to run one process. The only bad thing with this is that the + heavy and long-running tasks will run *in* the webserver, keeping the user + waiting each time some heavy lifting is needed as in for example processing + a video. This could lead to problems as an aborted connection will halt any + processing and since most front-end web servers *will* terminate your + connection if it doesn't get any response from the MediaGoblin WSGI + application in a while. + +2. As a separate process communicating with the MediaGoblin WSGI application + via a `broker`_. This offloads the heavy lifting from the MediaGoblin WSGI + application and users will be able to continue to browse the site while the + media is being processed in the background. + +.. _`broker`: http://docs.celeryproject.org/en/latest/getting-started/brokers/ +.. _`celery`: http://www.celeryproject.org/ + -In nearly every scenario, work on the Celery queue will need to -balance with the demands of other processes, and cannot proceed -synchronously. This is a particularly relevant problem if you use -MediaGoblin to host video content. Processing with Celery ought to be -operationally separate from the MediaGoblin application itself, this -simplifies management and support better workload distribution. +.. [#f-mediagoblin-wsgi-app] The MediaGoblin WSGI application is the part that + of MediaGoblin that processes HTTP requests. -Basically, if you're doing anything beyond a trivial workload, such as -image hosting for a small set of users, or have limited media types -such as "ASCII art" or icon sharing, you will need to run ``celeryd`` -as a separate process. +To launch Celery separately from the MediaGoblin WSGI application: -Build an :ref:`init script <init-script>` around the following -command:: +1. Make sure that the ``CELERY_ALWAYS_EAGER`` environment variable is unset or + set to ``false`` when launching the MediaGoblin WSGI application. +2. Start the ``celeryd`` main process with - CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd + .. code-block:: bash -Modify your existing MediaGoblin and application init scripts, if -necessary, to prevent them from starting their own ``celeryd`` -processes. + CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd .. _sentry: @@ -102,7 +116,7 @@ These are scripts provided by the MediaGoblin community: Debian * `GNU MediaGoblin init scripts - <https://github.com/jwandborg/mediagoblin-init-scripts>`_ + <https://github.com/joar/mediagoblin-init-scripts>`_ by `Joar Wandborg <http://wandborg.se>`_ Arch Linux diff --git a/docs/source/siteadmin/relnotes.rst b/docs/source/siteadmin/relnotes.rst index 6962dc5a..3542bdcb 100644 --- a/docs/source/siteadmin/relnotes.rst +++ b/docs/source/siteadmin/relnotes.rst @@ -11,6 +11,8 @@ Dedication along with this software. If not, see <http://creativecommons.org/publicdomain/zero/1.0/>. +.. _release-notes: + ============= Release Notes ============= @@ -19,6 +21,250 @@ This chapter has important information for releases in it. If you're upgrading from a previous release, please read it carefully, or at least skim over it. +0.6.1 +===== + +This is a short, bugfix release. + +**Do this to upgrade** + +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.6.1`` +2. Make sure to run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +This release switches the default terms of service to be off by +default and corrects some mistakes in the default terms of service. + +Turning the terms of service on is very easy, just set ``show_tos`` in +the ``[mediagoblin]`` section of your config to ``true``. + + +0.6.0 +===== + +**Do this to upgrade** + +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.6.0`` +2. Make sure to run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +That's it, probably! If you run into problems, don't hesitate to +`contact us <http://mediagoblin.org/pages/join.html>`_ +(IRC is often best). + +This tool has a lot of new tools for administrators, hence the +nickname "Lore of the Admin"! + +**New features:** + +- New tools to control how much users can upload, both as a general + user limit, or per file. + + You can set this with the following options in your mediagoblin + config file: `upload_limit` and `max_file_size`. Both are integers + in megabytes. + + There is an option to control how much each individual user can + upload too, though an interface for this is not yet exposed. See + the "uploaded" field on the core__users table. + +- MediaGoblin now contains an authentication plugin for ldap! You + can turn on the mediagoblin.plugins.ldap plugin to make use of + this. See the documentation: :ref:`ldap-plugin` + +- There's a new command line upload tool! At long last! See + `./bin/gmg addmedia --help` for info on how to use this. + +- There's now a terms of service document included in MediaGoblin. + It's turned on by default, but you can turn it off if you prefer, + just set the configuration option of `show_tos` in the [mediagoblin] + section of your config to false. + + Alternately, you can override the template for the terms of service + document to set up your own. + +- We have a lot of new administrative tooling features! + - There's a built-in privileges/permissions system now. + Administrators are given access to modifying these parameters + from a user administration panel. + - Users can submit reports about other problematic users or media + and administrators are given tools to resolve said reports and + ban/unban users if needed. + +- New version of video.js is included with MediaGoblin. Slight + amount of skinning to match the MediaGoblin look, otherwise also + uses the new default skin. + +Developer-oriented changes: + +- New developer tool for quickly setting up a development environment + in `devtools/make_example_database.sh`. Requires doing a checkout + of our other tool `mg_dev_environments <https://gitorious.org/mediagoblin/mg-dev-environments/>`_ + (probably in the parent Directory) though! +- A "foundations" framework has entered into the codebase. + This is mostly just relevant to coders, but it does mean that it's + much easier to add database structures that need some entries filled + automatically by default. +- Refactoring to the authentication code and the reprocessing code + + +0.5.1 +===== + +v0.5.1 is a bugfix release... the steps are the same as for 0.5.1. + +**Bugfixes:** + +- python 2.6 compatibility restored +- Fixed last release's release notes ;) + + +0.5.0 +===== + +**NOTE:** If using the API is important to you, we're in a state of +ransition towards a new API via the Pump API. As such, though the old +API still probably works, some changes have happened to the way oauth +works to make it more Pump-compatible. If you're heavily using +clients using the old API, you may wish to hold off on upgrading for +now. Otherwise, jump in and have fun! :) + +**Do this to upgrade** + +1. Make sure to run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + after upgrading. + +.. mention something about new, experimental configure && make support + +2. Note that a couple of things have changed with ``mediagoblin.ini``. First + we have a new Authentication System. You need to add + ``[[mediagoblin.plugins.basic_auth]]`` under the ``[plugins]`` section of + your config file. Second, media types are now plugins, so you need to add + each media type under the ``[plugins]`` section of your config file. + + +3. We have made a script to transition your ``mediagoblin_local.ini`` file for + you. This script can be found at: + + http://mediagoblin.org/download/0.5.0_config_converter.py + +If you run into problems, don't hesitate to +`contact us <http://mediagoblin.org/pages/join.html>`_ +(IRC is often best). + +**New features** + +* As mentioned above, we now have a plugable Authentication system. You can + use any combination of the multiple authentication systems + (:ref:`basic_auth-chapter`, :ref:`persona-chapter`, :ref:`openid-chapter`) + or write your own! +* Media types are now plugins! This means that new media types will + be able to do new, fancy things they couldn't in the future. +* We now have notification support! This allows you to subscribe to media + comments and to be notified when someone comments on your media. +* New reprocessing framework! You can now reprocess failed uploads, and + send already processed media back to processing to re-transcode or resize + media. +* Comment preview! +* Users now have the ability to change their email associated with their + account. +* New oauth code as we move closer to federation support. +* Experimental pyconfigure support for GNU-style configue and makefile + deployment. +* Database foundations! You can now pre-populate the database models. +* Way faster unit test run-time via in-memory database. +* All mongokit stuff has been cleaned up. +* Fixes for non-ascii filenames. +* The option to stay logged in. +* Mediagoblin has been upgraded to use the latest `celery <http://celeryproject.org/>`_ + version. +* You can now add jinja2 extensions to your config file to use in custom + templates. +* Fixed video permission issues. +* Mediagoblin docs are now hosted with multiple versions. +* We removed redundent tooltips from the STL media display. +* We are now using itsdangerous for verification tokens. + + +0.4.1 +===== + +This is a bugfix release for 0.4.0. This only implements one major +fix in the newly released document support which prevented the +"conversion via libreoffice" feature. + +If you were running 0.4.0 you can upgrade to v0.4.1 via a simple +switch and restarting mediagoblin/celery with no other actions. + +Otherwise, follow 0.4.0 instructions. + + +0.4.0 +===== + +**Do this to upgrade** + +1. Make sure to run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + after upgrading. +2. See "For Theme authors" if you have a custom theme. +3. Note that ``./bin/gmg theme assetlink`` is now just + ``./bin/gmg assetlink`` and covers both plugins and assets. + Keep on reading to hear more about new plugin features. +4. If you want to take advantage of new plugins that have statically + served assets, you are going to need to add the new "plugin_static" + section to your nginx config. Basically the following for nginx:: + + # Plugin static files (usually symlinked in) + location /plugin_static/ { + alias /srv/mediagoblin.example.org/mediagoblin/user_dev/plugin_static/; + } + + Similarly, if you've got a modified paste config, you may want to + borrow the app:plugin_static section from the default paste.ini + file. +5. We now use itsdangerous for sessions; if you had any references to + beaker in your paste config you can remove them. Again, see the + default paste.ini config +6. We also now use git submodules. Please do: + ``git submodule init && git submodule update`` + You will need to do this to use the new PDF support. + +**For theme authors** + +If you have your own theme or you have any "user modified templates", +please note the following: + +* mediagoblin/bits/ files above-content.html, body-end.html, + body-start.html now are renamed... they have underscores instead of + dashes in the filenames now :) +* There's a new file: ``mediagoblin/bits/frontpage_welcome.html``. + You can easily customize this to give a welcome page appropriate to + your site. + + +**New features** + +* PDF media type! +* Improved plugin system. More flexible, better documented, with a + new plugin authoring section of the docs. +* itsdangerous based sessions. No more beaker! +* New, experimental Piwigo-based API. This means you should be able + to use MediaGoblin with something like Shotwell. (Again, a word of + caution: this is *very experimental*!) +* Human readable timestamps, and the option to display the original + date of an image when available (available as the + "original_date_visible" variable) +* Moved unit testing system from nosetests to py.test so we can better + handle issues with sqlalchemy exploding with different database + configurations. Long story :) +* You can now disable the ability to post comments. +* Tags now can be up to length 255 characters by default. + + 0.3.3 ===== @@ -30,8 +276,8 @@ carefully, or at least skim over it. .. code-block:: ini - [plugins] - [[mediagoblin.plugins.geolocation]] + [plugins] + [[mediagoblin.plugins.geolocation]] If you have your own theme, you may need to make some adjustments to it as some theme related things may have changed in this release. If @@ -100,7 +346,19 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. **Do this to upgrade** -1. Make sure to run ``bin/gmg dbupdate`` after upgrading. + # directory of your mediagoblin install + cd /srv/mediagoblin.example.org + + # copy source for this release + git fetch + git checkout tags/v0.3.2 + + # perform any needed database updates + bin/gmg dbupdate + + # restart your servers however you do that, e.g., + sudo service mediagoblin-paster restart + sudo service mediagoblin-celeryd restart **New features** diff --git a/docs/source/siteadmin/theming.rst b/docs/source/siteadmin/theming.rst index 98ec6de7..11ae3875 100644 --- a/docs/source/siteadmin/theming.rst +++ b/docs/source/siteadmin/theming.rst @@ -51,7 +51,7 @@ want to install this theme! Don't worry, it's fairly painless. 5. Link the assets so that they can be served by your web server:: - $ ./bin/gmg theme assetlink + $ ./bin/gmg assetlink .. Note:: |