diff options
Diffstat (limited to 'docs/source')
26 files changed, 1539 insertions, 394 deletions
diff --git a/docs/source/api/activities.rst b/docs/source/api/activities.rst new file mode 100644 index 00000000..cbbd1fab --- /dev/null +++ b/docs/source/api/activities.rst @@ -0,0 +1,208 @@ +.. MediaGoblin Documentation + + Written in 2015 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + +========== +Activities +========== + +.. contents:: Sections + :local: + +GNU MediaGoblin uses `Activity Streams 1.0 <http://activitystrea.ms>`_ JSON +format to represent activities or events that happen. There are several +components to Activity Streams. + +Objects +======= +These represent "things" in MediaGoblin such as types of media, comments, collections +of other objects, etc. There are attributes all objects have and some attributes which +are specific to certain objects. + +Example +------- +a representation of an image object:: + + { + "id": "https://gmg.server.tld/api/image/someidhere", + "objectType": "image", + "content": "My lovely image", + "image": { + "url": "https://gmg.server.tld/mgoblin_media/media_entries/23/some_image.jpg", + "height": 1000, + "width": 500 + }, + "author": { + "id": "acct:someone@gmg.server.tld" + } + } + +This has both attributes which are on all objects (e.g. ``objectType`` and ``id``) +and attributes which are on only images (e.g. ``image``). + +Activities +========== +This is something which happens such as: commenting on an image, uploading an image, etc. +these always have a ``verb`` which describes what kind of activity it is and they always have +an ``object`` associated with them. + +Example +------- +A activity which describes the event of posting a new image:: + + { + "id": "https://gmg.server.tld/api/activity/someidhere", + "verb": "post", + "object": { + "id": "https://gmg.server.tld/api/comment/someid", + "objectType": "comment", + "content": "What a wonderful picture you have there!", + "inReplyTo": { + "id": "https://gmg.server.tld/api/image/someidhere" + } + }, + "author": { + "id": "acct:someone@gmg.server.tld" + } + } + +Collections +=========== + +These are ordered lists which contain objects. Currently in GNU MediaGoblin they are used +to represent "albums" or collections of media however they can represent anything. They will +be used in the future to represent lists/groups of users which you can send activities to. + +Example +^^^^^^^ +A collection which contains two images:: + + { + "id": "https://gmg.server.tld/api/collection/someidhere", + "totalItems": 2, + "url": "http://gmg.server.tld/u/someone/collection/cool-album/", + "items": [ + { + "id": "https://gmg.server.tld/api/image/someidhere", + "objectType": "image", + "content": "My lovely image", + "image": { + "url": "https://gmg.server.tld/mgoblin_media/media_entries/23/some_image.jpg", + "height": 1000, + "width": 500 + }, + "author": { + "id": "acct:someone@gmg.server.tld" + } + }, + { + "id": "https://gmg.server.tld/api/image/someother", + "objectType": "image", + "content": "Another image for you", + "image": { + "url": "https://gmg.server.tld/mgoblin_media/media_entries/24/some_other_image.jpg", + "height": 1000, + "width": 500 + }, + "author": { + "id": "acct:someone@gmg.server.tld" + } + } + ] + } + +Feeds +----- + +There are several feeds which can be read and posted to as part of the API. Some +of the feeds are still a work in progress however a lot of them are present for +compatibility. + +They also support certain GET parameters which allow you to control the stream. +These are: + ++-------------+----------+----------+----------------------------------+ +| Parameter | Default | Limit | Description | ++=============+==========+==========+==================================+ +| count | 20 | 200 | Number of activities to return | ++-------------+----------+----------+----------------------------------+ +| offset | 0 | No limit | Offset of collection | ++-------------+----------+----------+----------------------------------+ + +.. warning:: + Activities are added to the beginning of collection so using count and + offset is a bad way of doing pages. + +.. important:: + Due to the way we're currently doing deletes in MediaGoblin some activities + are broken and are skipped. This means the number you specify in the count + is NOT always the number of activities returned. + + +Inbox +^^^^^ + +**Endpoint:** `/api/user/<username>/inbox` + +This feed can be read by user to see what media has been sent to them. +MediaGoblin currently doesn't have the ability to sent media to anyone +as all media is public, all media on that instance should show up in the +users inbox. + +There are also subsets of the inbox which are: + +Major +""""" +**Endpoint:** ``/api/user/<username>/inbox/major`` + +This contains all major changes such as new objects being posted. Currently +comments exist in this feed, in the future they will be moved to the minor feed. + +Minor +""""" +**Endpoint:** ``/api/user/<username>/inbox/minor`` + +This contains minor changes such as objects being updated or deleted. This feed +should have comments in it, currently they are listed under major, in the future +they will exist in this endpoint. + +Direct +"""""" +**Endpoint:** ``/api/user/<username>/inbox/direct`` + +Currently this is just a mirror of the regular inbox for compatibility with +pump.io. In the future this will contain all objects specifically addressed to +the user. + +Direct major +"""""""""""" +**Endpoint:** ``/api/user/<username>/inbox/direct/major`` + +Currently this is just a mirror of the major inbox for compatibility with +pump.io. In the future this will contain all major activities which are +specifically addressed to the user. + +Direct minor +"""""""""""" +**Endpoint:** ``/api/user/<username>/inbox/direct/minor`` + +Currently this is just a mirror of the minor inbox for compatibility with +pump.io. In the future this will contain all minor activities which are +specifically addressed to the user. + +Feed (outbox) +^^^^^^^^^^^^^ +**Endpoint:** ``/api/user/<username>/feed`` + +This is where the client should post new activities. It can be read by the +user to see what they have posted. This will only contain content they have +authored or shared (sharing will come in the future). diff --git a/docs/source/api/client_register.rst b/docs/source/api/authentication.rst index 08f92c47..2db4a7bd 100644 --- a/docs/source/api/client_register.rst +++ b/docs/source/api/authentication.rst @@ -1,6 +1,6 @@ .. MediaGoblin Documentation - Written in 2011, 2012 by MediaGoblin contributors + Written in 2015 by MediaGoblin contributors To the extent possible under law, the author(s) have dedicated all copyright and related and neighboring rights to this software to @@ -11,11 +11,17 @@ Dedication along with this software. If not, see <http://creativecommons.org/publicdomain/zero/1.0/>. -==================== +================== +API Authentication +================== + +.. contents:: Sections + :local: + Registering a Client ==================== -To use the GNU MediaGoblin API you need to use the dynamic client registration. This has been adapted from the `OpenID specification <https://openid.net/specs/openid-connect-registration-1_0.html>`_, this is the only part of OpenID that is being used to serve the purpose to provide the client registration which is used in OAuth. +To use the GNU MediaGoblin API you need to use the dynamic client registration. This has been adapted from the `OpenID specification <https://openid.net/specs/openid-connect-registration-1_0.html>`_, this is the only part of OpenID that is being used to serve the purpose to provide the client registration which is used in OAuth. The endpoint is ``/api/client/register`` @@ -31,7 +37,7 @@ client_secret **update only** - This should only be used updating client information, this is the client_secret given when you register contacts - **optional** - This a space seporated list of email addresses to contact of people responsible for the client + **optional** - This a space separated list of email addresses to contact of people responsible for the client application_type **required** - This is the type of client you are making, this must be either *web* or *native* @@ -39,15 +45,15 @@ application_type application_name **optional** - This is the name of your client -logo_url - **optional** - This is a URL of the logo image for your client +logo_uri + **optional** - This is a URI of the logo image for your client redirect_uri - **optional** - This is a space seporated list of pre-registered URLs for use at the Authorization Server + **optional** - This is a space separated list of pre-registered URLs for use at the Authentication Server Response --------- +^^^^^^^^ You will get back a response: @@ -60,12 +66,11 @@ client_secret expires_at This is time that the client credentials expire. If this is 0 the client registration does not expire. -======= -Example -======= +Examples +-------- Register Client ---------------- +^^^^^^^^^^^^^^^ To register a client for the first time, this is the minimum you must supply:: @@ -84,7 +89,7 @@ A Response will look like:: Updating Client ---------------- +^^^^^^^^^^^^^^^ Using the response we got above we can update the information and add new information we may have opted not to supply:: @@ -93,8 +98,8 @@ Using the response we got above we can update the information and add new inform "client_id": "vwljdhUMhhNbdKizpjZlxv", "client_secret": "hJtfhaQzgKerlLVdaeRAgmbcstSOBLRfgOinMxBCHcb", "application_type": "web", - "application_name": "MyClient!", - "logo_url": "https://myclient.org/images/my_logo.png", + "application_name": "MyClient!", + "logo_uri": "https://myclient.org/images/my_logo.png", "contacts": "myemail@someprovider.com another_developer@provider.net", } @@ -107,9 +112,8 @@ The response will just return back the client_id and client_secret you sent:: } -====== -Errors -====== +Possible Registration Errors +---------------------------- There are a number of errors you could get back, This explains what could cause some of them: @@ -129,7 +133,7 @@ client_id is required to update. When you try and update you need to specify the client_id, this will be what you were given when you initially registered the client. client_secret is required to update. - When you try to update you need to specify the client_secrer, this will be what you were given when you initially register the client. + When you try to update you need to specify the client_secret, this will be what you were given when you initially register the client. Unauthorized. This is when you are trying to update however the client_id and/or client_secret you have submitted are incorrect. @@ -140,19 +144,38 @@ Only set client_id for update. Only set client_secret for update. This should only be given when you update. -Logo URL <url> is not a valid URL +Logo URL <URL> is not a valid URL This is when the URL specified did not meet the validation. contacts must be a string of space-separated email addresses. - ``contacts`` should be a string (not a list), ensure each email is seporated by a space + ``contacts`` should be a string (not a list), ensure each email is separated by a space Email <email> is not a valid email This is when you have submitted an invalid email address redirect_uris must be space-separated URLs. - ``redirect_uris`` should be a string (not a list), ensure each URL is seporated by a space + ``redirect_uris`` should be a string (not a list), ensure each URL is separated by a space URI <URI> is not a valid URI This is when your URI is invalid. - +OAuth +===== + +GNU MediaGoblin uses OAuth1 to authenticate requests to the API. There are many +libraries out there for OAuth1, you're likely not going to have to do much. There +is a library for the GNU MediaGoblin called `PyPump <https://github.com/xray7224/PyPump>`_. +We are not using OAuth2 as we want to stay completely compatible with pump.io. + + +Endpoints +--------- + +These are the endpoints you need to use for the OAuth requests: + +`/oauth/request_token` is for getting the request token. + +`/oauth/authorize` is to send the user to to authorize your application. + +`/oauth/access_token` is for getting the access token to use in requests. + diff --git a/docs/source/api/oauth.rst b/docs/source/api/oauth.rst deleted file mode 100644 index 003ad492..00000000 --- a/docs/source/api/oauth.rst +++ /dev/null @@ -1,36 +0,0 @@ -.. MediaGoblin Documentation - - Written in 2011, 2012 by MediaGoblin contributors - - To the extent possible under law, the author(s) have dedicated all - copyright and related and neighboring rights to this software to - the public domain worldwide. This software is distributed without - any warranty. - - You should have received a copy of the CC0 Public Domain - Dedication along with this software. If not, see - <http://creativecommons.org/publicdomain/zero/1.0/>. - -============== -Authentication -============== - -GNU MediaGoblin uses OAuth1 to authenticate requests to the API. There are many -libraries out there for OAuth1, you're likely not going to have to do much. There -is a library for the GNU MediaGoblin called `PyPump <https://github.com/xray7224/PyPump>`_. -We are not using OAuth2 as we want to stay completely compatable with GNU MediaGoblin. - - -We use :doc:`client_register` to get the client ID and secret. - -Endpoints ---------- - -These are the endpoints you need to use for the oauth requests: - -`/oauth/request_token` is for getting the request token. - -`/oauth/authorize` is to send the user to to authorize your application. - -`/oauth/access_token` is for getting the access token to use in requests. - diff --git a/docs/source/api/objects.rst b/docs/source/api/objects.rst new file mode 100644 index 00000000..854febe1 --- /dev/null +++ b/docs/source/api/objects.rst @@ -0,0 +1,229 @@ +.. MediaGoblin Documentation + + Written in 2015 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + +.. info:: Currently only image uploading is supported. + +======= +Objects +======= + +Using any the APIs mentioned in this document you will required +:doc:`authentication`. There are many ways to interact with objects, some of +which aren't supported yet by MediaGoblin such as liking or sharing objects +however you can interact with them by updating them, deleting them and +commenting on them. + +Posting Objects +--------------- + +For the most part you should be able to post objects by creating the JSON +representation of the object on an activity and posting that to the user's +feed (outbox). Images however are slightly different and there are more steps +to it as you might imagine. + +Using posting a comment as an example, I'll show you how to post the object +to GNU MediaGoblin or pump.io. I first need to create the JSON representation +of the activity with the object but without the ID, URL, published or updated +timestamps or any other data the server creates. My activity comment is:: + + { + "verb": "post", + "object": { + "objectType": "comment", + "content": "This is my comment to be posted", + "inReplyTo": { + "id": "https://<server>/api/image/1" + } + } + } + +This should be posted to the users feed (outbox) which you can find out about +:doc:`activities`. You will get back the full activity containing all of +attributes including ID, URLs, etc. + +Posting Media +~~~~~~~~~~~~~ + +Posting media is a special case from posting all other objects. This is because +you need to submit more than just the JSON image representation, you need to +actually subject the image itself. There is also strange behavior around media +postings where if you want to give the media you're posting a title or +description you need to perform an update too. A full media posting in order of +steps to do is as follows: + +1) Uploads the data to the server +2) Post media to feed +3) Update media to have title, description, license, etc. (optional) + +This could be condensed into a 2-step process however this would need to happen +upstream. If you would like to contribute to changing this upstream there is +an issue open: https://github.com/e14n/pump.io/issues/657 + +To upload media you should use the URL `/api/user/<username>/uploads`. + +A POST request should be made to the media upload URL submitting at least two +headers: + +* `Content-Type` - This being a valid MIME type for the media. +* `Content-Length` - size in bytes of the media. + +The media data should be submitted as POST data to the image upload URL. +You will get back a JSON encoded response which will look similar to:: + + { + "updated": "2014-01-11T09:45:48Z", + "links": { + "self": { + "href": "https://<server>/image/4wiBUV1HT8GRqseyvX8m-w" + } + }, + "fullImage": { + "url": "https://<server>//uploads/<username>/2014/1/11/V3cBMw.jpg", + "width": 505, + "height": 600 + }, + "replies": { + "url": "https://<server>//api/image/4wiBUV1HT8GRqseyvX8m-w/replies" + }, + "image": { + "url": "https://<server>/uploads/<username>/2014/1/11/V3cBMw_thumb.jpg", + "width": 269, + "height": 320 + }, + "author": { + "preferredUsername": "<username>", + "displayName": "<username>", + "links": { + "activity-outbox": { + "href": "https://<server>/api/user/<username>/feed" + }, + "self": { + "href": "https://<server>/api/user/<username>/profile" + }, + "activity-inbox": { + "href": "https://<server>/api/user/<username>/inbox" + } + }, + "url": "https://<server>/<username>", + "updated": "2013-08-14T10:01:21Z", + "id": "acct:<username>@<server>", + "objectType": "person" + }, + "url": "https://<server>/<username>/image/4wiBUV1HT8GRqseyvX8m-w", + "published": "2014-01-11T09:45:48Z", + "id": "https://<server>/api/image/4wiBUV1HT8GRqseyvX8m-w", + "objectType": "image" + } + +The main things in this response is `fullImage` which contains `url` (the URL +of the original image - i.e. fullsize) and `image` which contains `url` (the URL +of a thumbnail version). + +.. warning:: Media which have been uploaded but not submitted to a feed will + periodically be deleted. + +Once you've got the image object back you will need to submit the post +activity to the feed. This is exactly the same process as posting any other +object described above. You create a post activity and post that to the feed +(outbox) endpoint. The post activity looks like:: + + { + "verb": "post", + "object": { + "id": "https://<server>/api/image/4wiBUV1HT8GRqseyvX8m-w", + "objectType": "image" + } + } + +You will get back the full activity, unlike above however if you wish to +submit `displayName` (title) or `content` (description) information you need +to create an update activity and post that to the feed after you have posted +the image. An update activity would look like:: + + { + "verb": "update", + "object": { + "id": "https://<server>/api/image/4wiBUV1HT8GRqseyvX8m-w", + "displayName": "This is my title", + "content": "This is my description", + "objectType": "image" + } + } + +Updating Objects +---------------- + +If you would like to edit or update an object you can do so by submitting an +update activity. An update to a comment might look like:: + + { + "verb": "update", + "object": { + "id": "https://<server>/api/comment/1", + "objectType": "comment", + "content": "This is my new updated comment!" + } + } + +This should be posted to the feed (outbox). You will get back the full update +activity in response. + +Deleting Objects +---------------- + +Objects can be deleted by submitting a delete activity to the feed. A delete +object for a comment looks like:: + + { + "verb": "delete", + "object": { + "id": "https://<server>/api/comment/id", + "objectType": "comment" + } + } + +You should get the full delete activity in response. + +.. warning:: + While deletion works, currently because of the way deletion is implemented + deletion either via the API or the web UI causes any activities to be broken + and will be skipped and inaccessible. A migration to remove the broken + activities will come in a future release when soft-deletion has been + implemented. + +Posting Comments +---------------- + +Comments currently can only be on media objects, this however will change in +future versions of MediaGoblin to be inline with pump.io and Activity Streams +1.0 which allow comments to be on any object including comments themselves. + +If you want to submit a comment on an object it's very easy, it's just like +posting any other object except you use the `inReplyTo` attribute which +specifies the object you are commenting on. The `inReplyTo` needs to contain +the object or specifically the ID of it. + +Example of comment on an image:: + + { + "verb": "post", + "object": { + "content": "My comment here", + "inReplyTo": { + "id": "https://<server>/api/image/72" + } + } + } + +This should be posted to a feed and you will get back the full activity object +as with any other object posting. diff --git a/docs/source/conf.py b/docs/source/conf.py index 0b2bccac..0f53e4cf 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -219,7 +219,7 @@ latex_documents = [ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ - ('index', 'gnumediagoblin', u'GNU MediaGoblin Documentation', + ('index', 'mediagoblin', u'GNU MediaGoblin Documentation', [u'Chris Webber, et al'], 1) ] @@ -233,7 +233,7 @@ man_pages = [ # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ - ('index', 'gnumediagoblin', u'GNU MediaGoblin Documentation', u'gnumediagoblin', + ('index', 'mediagoblin', u'GNU MediaGoblin Documentation', u'mediagoblin', 'GNU MediaGoblin', 'Media sharing web application.', 'Miscellaneous'), ] diff --git a/docs/source/devel/codebase.rst b/docs/source/devel/codebase.rst index 122a3297..068c1873 100644 --- a/docs/source/devel/codebase.rst +++ b/docs/source/devel/codebase.rst @@ -44,7 +44,7 @@ development. What's where ============ -After you've run checked out mediagoblin and followed the virtualenv +After you've run checked out MediaGoblin and followed the virtualenv instantiation instructions, you're faced with the following directory tree:: @@ -86,8 +86,8 @@ As you can see, all the code for GNU MediaGoblin is in the Here are some interesting files and what they do: -:routing.py: maps url paths to views -:views.py: views handle http requests +:routing.py: maps URL paths to views +:views.py: views handle HTTP requests :forms.py: wtforms stuff for this submodule You'll notice that there are several sub-directories: tests, @@ -97,7 +97,7 @@ templates, auth, submit, ... ``templates`` holds all the templates for the output. -``auth`` and ``submit`` are modules that enacpsulate authentication +``auth`` and ``submit`` are modules that encapsulate authentication and media item submission. If you look in these directories, you'll see they have their own ``routing.py``, ``view.py``, and forms.py in addition to some other code. @@ -123,15 +123,15 @@ Software Stack for unit tests * `virtualenv <http://www.virtualenv.org/>`_: for setting up an - isolated environment to keep mediagoblin and related packages + isolated environment to keep MediaGoblin and related packages (potentially not required if MediaGoblin is packaged for your distro) * Data storage * `SQLAlchemy <http://sqlalchemy.org/>`_: SQL ORM and database - interaction library for Python. Currently we support sqlite and - postgress as backends. + interaction library for Python. Currently we support SQLite and + PostgreSQL as backends. * Web application @@ -158,10 +158,10 @@ Software Stack * `Markdown (for python) <http://pypi.python.org/pypi/Markdown>`_: implementation of `Markdown <http://daringfireball.net/projects/markdown/>`_ - text-to-html tool to make it easy for people to write richtext + text-to-html tool to make it easy for people to write rich text comments, descriptions, and etc. - * `lxml <http://lxml.de/>`_: nice xml and html processing for + * `lxml <http://lxml.de/>`_: nice XML and HTML processing for python. * Media processing libraries @@ -174,10 +174,10 @@ Software Stack future, probably audio too. * `chardet <http://pypi.python.org/pypi/chardet>`_: (Optional, for - ascii art hosting sites only) Used to make ascii art thumbnails. + ASCII art hosting sites only) Used to make ASCII art thumbnails. * Front end - * `JQuery <http://jquery.com/>`_: for groovy JavaScript things + * `jQuery <http://jquery.com/>`_: for groovy JavaScript things diff --git a/docs/source/devel/migrations.rst b/docs/source/devel/migrations.rst index 16c02b04..ec0f7d29 100644 --- a/docs/source/devel/migrations.rst +++ b/docs/source/devel/migrations.rst @@ -28,35 +28,35 @@ every migration is run with dbupdate. There's a few things you need to know: -- We use `sqlalchemy-migrate +- We use `Alembic <https://bitbucket.org/zzzeek/alembic>`_ to run + migrations. We also make heavy use of the + `branching model <http://alembic.readthedocs.org/en/latest/branches.html>`_ + for our plugins. Every plugin gets its own migration branch. +- We used to use `sqlalchemy-migrate <http://code.google.com/p/sqlalchemy-migrate/>`_. See `their docs <https://sqlalchemy-migrate.readthedocs.org/>`_. -- `Alembic <https://bitbucket.org/zzzeek/alembic>`_ might be a better - choice than sqlalchemy-migrate now or in the future, but we - originally decided not to use it because it didn't have sqlite - support. It's not clear if that's changed. + sqlalchemy-migrate is now only kept around for legacy migrations; + don't add any more! But some users are still using older databases, + and we need to provide the intermediary "old" migrations for a while. - SQLAlchemy has two parts to it, the ORM and the "core" interface. We DO NOT use the ORM when running migrations. Think about it: the ORM is set up with an expectation that the models already reflect a - certain pattern. But if a person is moving from their old patern + certain pattern. But if a person is moving from their old pattern and are running tools to *get to* the current pattern, of course their current database structure doesn't match the state of the ORM! -- How to write migrations? Maybe there will be a tutorial here in the - future... in the meanwhile, look at existing migrations in - `mediagoblin/db/migrations.py` and look in - `mediagoblin/tests/test_sql_migrations.py` for examples. -- Common pattern: use `inspect_table` to get the current state - of the table before we run alterations on it. -- Make sure you set the RegisterMigration to be the next migration in - order. -- What happens if you're adding a *totally new* table? In this case, - you should copy the table in entirety as it exists into - migrations.py then create the tables based off of that... see - add_collection_tables. This is easier than reproducing the SQL by - hand. -- If you're writing a feature branch, you don't need to keep adding - migrations every time you change things around if your database - structure is in flux. Just alter your migrations so that they're - correct for the merge into master. + Anyway, Alembic has its own conventions for migrations; follow those. +- Alembic's documentation is pretty good; you don't need to worry about + setting up the migration environment or the config file so you can + skip those parts. You can start at the + `Create a Migration Script <http://alembic.readthedocs.org/en/latest/tutorial.html#create-a-migration-script>`_ + section. +- *Users* should only use `./bin/gmg dbupdate`. However, *developers* + may wish to use the `./bin/gmg alembic` subcommand, which wraps + alembic's own command line interface. Alembic has some tools for + `autogenerating migrations <http://alembic.readthedocs.org/en/latest/autogenerate.html>`_, + and they aren't perfect, but they are helpful. (You can pass in + `./bin/gmg alembic --with-plugins revision --autogenerate` if you need + to include plugins in the generated output; see the + :ref:`plugin database chapter <plugin-database-chapter>` for more info.) That's it for now! Good luck! diff --git a/docs/source/devel/originaldesigndecisions.rst b/docs/source/devel/originaldesigndecisions.rst index 2843870c..b60db776 100644 --- a/docs/source/devel/originaldesigndecisions.rst +++ b/docs/source/devel/originaldesigndecisions.rst @@ -38,7 +38,7 @@ Chris and Will on "Why GNU MediaGoblin": .. figure:: ../_static/goblin.png :alt: Cute goblin with a beret. - *Figure 1: Cute goblin with a beret. llustrated by Chris + *Figure 1: Cute goblin with a beret. Illustrated by Chris Webber* .. figure:: ../_static/snugglygoblin.png @@ -61,7 +61,7 @@ Chris and Will on "Why GNU MediaGoblin": 1. "GNU MediaGoblin" is the name we're going to use in all official capacities: web site, documentation, press releases, ... - 2. In casual conversation, it's ok to use more casual names. + 2. In casual conversation, it's OK to use more casual names. 3. If you're writing about the project, we ask that you call it GNU MediaGoblin. @@ -113,7 +113,7 @@ Why WSGI Minimalism Chris Webber on "Why WSGI Minimalism": If you notice in the technology list I list a lot of components - that are very "django-like", but not actually `Django`_ + that are very "Django-like", but not actually `Django`_ components. What can I say, I really like a lot of the ideas in Django! Which leads to the question: why not just use Django? @@ -176,7 +176,7 @@ Chris Webber on "Why MongoDB": ideal universe where everyone ran servers out of their own housing. As a memory-mapped database, MongoDB is pretty hungry, so actually I spent a lot of time debating whether the inability - to scale down as nicely as something like SQL has with sqlite + to scale down as nicely as something like SQL has with SQLite meant that it was out. But I decided in the end that I really want MongoDB, not for @@ -199,7 +199,7 @@ Chris Webber on "Why MongoDB": Being able to just dump media-specific information in a media_data - hashtable is pretty great, and even better is having a plugin + hash table is pretty great, and even better is having a plugin system where you can just let plugins have their own entire key-value space cleanly inside the document that doesn't interfere with anyone else's stuff. If we were to let plugins to deposit diff --git a/docs/source/devel/storage.rst b/docs/source/devel/storage.rst index 215f9579..46e163de 100644 --- a/docs/source/devel/storage.rst +++ b/docs/source/devel/storage.rst @@ -21,7 +21,7 @@ are: + **public_store:** After your media goes through processing it gets moved to the public store. This is also a StorageInterface - implelementation, and is for stuff that's intended to be seen by + implementation, and is for stuff that's intended to be seen by site visitors. The workbench @@ -43,11 +43,9 @@ Static assets / staticdirect ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ On top of all that, there is some static media that comes bundled with your -application. This stuff is kept in: +application. This stuff is kept in ``mediagoblin/static/``. - mediagoblin/static/ - -These files are for mediagoblin base assets. Things like the CSS files, +These files are for MediaGoblin base assets. Things like the CSS files, logos, etc. You can mount these at whatever location is appropriate to you (see the direct_remote_path option in the config file) so if your users are keeping their static assets at http://static.mgoblin.example.org/ but @@ -55,7 +53,7 @@ their actual site is at http://mgoblin.example.org/, you need to be able to get your static files in a where-it's-mounted agnostic way. There's a "staticdirector" attached to the request object. It's pretty easy to use; just look at this bit taken from the -mediagoblin/templates/mediagoblin/base.html main template: +mediagoblin/templates/mediagoblin/base.html main template:: <link rel="stylesheet" type="text/css" href="Template:Request.staticdirect('/css/extlib/text.css')"/> @@ -76,9 +74,7 @@ So, the StorageInterface! So, the public and queue stores both use StorageInterface implementations ... but what does that mean? It's not too hard. -Open up: - - mediagoblin/storage.py +Open up ``mediagoblin/storage.py``. In here you'll see a couple of things. First of all, there's the StorageInterface class. What you'll see is that this is just a very simple @@ -95,12 +91,12 @@ also the default storage system used. As expected, this stores files locally on your machine. There's also a CloudFileStorage system. This provides a mapping to -[OpenStack's swift http://swift.openstack.org/] storage system (used by +[OpenStack's Swift http://swift.openstack.org/] storage system (used by RackSpace Cloud files and etc). Between these two examples you should be able to get a pretty good idea of how to write your own storage systems, for storing data across your -beowulf cluster of radioactive monkey brains, whatever. +Beowulf cluster of radioactive monkey brains, whatever. Writing code to store stuff ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -112,10 +108,10 @@ https://docs.djangoproject.com/en/dev/ref/files/storage/]... with some differences. Basically, you access files on "file paths", which aren't exactly like -unix file paths, but are close. If you wanted to store a file on a path -like dir1/dir2/filename.jpg you'd actually write that file path like: +Unix file paths, but are close. If you wanted to store a file on a path +like dir1/dir2/filename.jpg you'd actually write that file path like:: -['dir1', 'dir2', 'filename.jpg'] + ['dir1', 'dir2', 'filename.jpg'] This way we can be *sure* that each component is actually a component of the path that's expected... we do some filename cleaning on each component. diff --git a/docs/source/index.rst b/docs/source/index.rst index 3ead6136..fbc57154 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -20,7 +20,7 @@ GNU MediaGoblin is a platform for sharing photos, video and other media in an environment that respects our freedom and independence. This is a Free Software project. It is built by contributors for all -to use and enjoy. If you're intrested in contributing, see `the wiki +to use and enjoy. If you're interested in contributing, see `the wiki <http://wiki.mediagoblin.org/>`_ which has pages that talk about the ways someone can contribute. @@ -78,6 +78,7 @@ This guide covers writing new GNU MediaGoblin plugins. pluginwriter/database pluginwriter/api pluginwriter/tests + pluginwriter/hooks pluginwriter/media_type_hooks pluginwriter/authhooks @@ -96,6 +97,25 @@ This chapter contains various information for developers. devel/migrations +Part 5: Pump API +================ + +This chapter covers MediaGoblin's `Pump API +<https://github.com/e14n/pump.io/blob/master/API.md>`_ support. (A +work in progress; full federation is not supported at the moment, but +media uploading works! You can use something like +`PyPump <http://pypump.org>`_ +to write MediaGoblin applications.) + +.. toctree:: + :maxdepth: 1 + + api/authentication + api/activities + api/objects + + + Indices and tables ================== diff --git a/docs/source/plugindocs/oauth.rst b/docs/source/plugindocs/oauth.rst deleted file mode 100644 index 0685c9d1..00000000 --- a/docs/source/plugindocs/oauth.rst +++ /dev/null @@ -1 +0,0 @@ -.. include:: ../../../mediagoblin/plugins/oauth/README.rst diff --git a/docs/source/pluginwriter/api.rst b/docs/source/pluginwriter/api.rst index 29adb691..03761a38 100644 --- a/docs/source/pluginwriter/api.rst +++ b/docs/source/pluginwriter/api.rst @@ -42,7 +42,7 @@ exist yet: what to do then? The plan at present is that we are adding hooks as people need them, with community discussion. If you find that you need a hook and MediaGoblin at present doesn't provide it at present, please -`http://mediagoblin.org/pages/join.html <talk to us>`_! We'll +`talk to us <http://mediagoblin.org/pages/join.html>`_! We'll evaluate what to do from there. @@ -83,7 +83,7 @@ can use a shortcut to get your plugin's config section:: >>> floobie_dir = floobie_config['floobie_dir'] A tip: you have access to the `%(here)s` variable in your config, -which is the directory that the user's mediagoblin config is running +which is the directory that the user's MediaGoblin config is running out of. So for example, your plugin may need a "floobie" directory to store floobs in. You could give them a reasonable default that makes use of the default `user_dev` location, but allow users to override @@ -92,7 +92,7 @@ it, like so:: [plugin_spec] floobie_dir = string(default="%(here)s/user_dev/floobs/") -Note, this is relative to the user's mediagoblin config directory, +Note, this is relative to the user's MediaGoblin config directory, *not* your plugin directory! @@ -178,7 +178,7 @@ Adding static resources ----------------------- It's possible to add static resources for your plugin. Say your -plugin needs some special javascript and images... how to provide +plugin needs some special JavaScript and images... how to provide them? Then how to access them? MediaGoblin has a way! @@ -230,7 +230,7 @@ the MediaGoblin repository. WTForms hooks +++++++++++++ -We haven't totally settled on a way to tranform wtforms form objects, +We haven't totally settled on a way to transform wtforms form objects, but here's one way. In your view:: from mediagoblin.foo.forms import SomeForm diff --git a/docs/source/pluginwriter/authhooks.rst b/docs/source/pluginwriter/authhooks.rst index 9721d729..66d9b9da 100644 --- a/docs/source/pluginwriter/authhooks.rst +++ b/docs/source/pluginwriter/authhooks.rst @@ -44,7 +44,7 @@ This hook is called in ``mediagoblin.auth.views`` in both the ``login`` and if :ref:`basic_auth-chapter` is not enabled, the user will be redirected to the correct login and registration views for your plugin. -The code assumes that it can generate a valid url given +The code assumes that it can generate a valid URL given ``mediagoblin.plugins.{{ your_plugin_here }}.login`` and ``mediagoblin.plugins.{{ your_plugin_here }}.register``. This is only needed if you will not be using the ``login`` and ``register`` views in @@ -82,5 +82,5 @@ the ``stored_hash`` and return either ``True`` or ``False``. This hook is called in ``mediagoblin.auth.tools.check_login_simple``. It is called if a user is not found and should do something that takes the same amount -of time as your ``check_password`` function. This is to help prevent timining +of time as your ``check_password`` function. This is to help prevent timing attacks. diff --git a/docs/source/pluginwriter/database.rst b/docs/source/pluginwriter/database.rst index 603a19eb..90c2cee3 100644 --- a/docs/source/pluginwriter/database.rst +++ b/docs/source/pluginwriter/database.rst @@ -24,7 +24,7 @@ Accessing Existing Data ======================= If your plugin wants to access existing data, this is quite -straight forward. Just import the appropiate models and use +straight forward. Just import the appropriate models and use the full power of SQLAlchemy. Take a look at the (upcoming) database section in the Developer's Chapter. @@ -61,7 +61,39 @@ Here's a simple one: MODELS = [MediaSecurity] -That's it. +Next, you need to make an initial migration. MediaGoblin uses +`Alembic's branching model <http://alembic.readthedocs.org/en/latest/branches.html>`_ +to handle plugins adding their own content. As such, when you are +adding a new plugin, you need to add an initial migration adding +the existing models, and migrate from there. + +You'll need to make a `migrations` subdirectory for migrations and put +your migrations there. If you want to look at some example +migrations, look at `mediagoblin/media_types/image/migrations/`, +especially the "initial" migration. (Plugin authors with plugins that +existed prior to the alembic switchover: you might notice how it +checks for the table and skips the migration if it already exists. +Plugin authors writing brand new plugins, post-Alembic migration +switchover, do not need to do this.) + +Unfortunately, these migrations are a bit tedious to write. +Fortunately, Alembic can do much of the work for us! After adding the +models.py file, run this command (switching out YOUR_PLUGIN_NAME of +course):: + + ./bin/gmg alembic --with-plugins revision \ + --splice --autogenerate \ + --branch-label YOUR_PLUGIN_NAME_plugin \ + -m "YOUR_PLUGIN_NAME plugin initial migration" + +(Note that `--with-plugins` *must* come before any alembic subcommand... +this is a quirk related to the way we handle alembic command dispatching +with the gmg subcommand!) + +This will dump your migration into "the wrong place" (it'll dump it +into the MediaGoblin core migrations directory), so you should move it +to your plugin's migrations directory. Open the file and make adjustments +accordingly! Some notes: @@ -78,37 +110,41 @@ Some notes: Changing the Database Schema Later ================================== -If your plugin is in use and instances use it to store some -data, changing the database design is a tricky thing. +If your plugin is in use and instances use it to store some data, +changing the database design is tricky and must be done with care, +but is not impossible. -1. Make up your mind how the new schema should look like. -2. Change ``models.py`` to contain the new schema. Keep a - copy of the old version around for your personal - reference later. -3. Now make up your mind (possibly using your old and new - ``models.py``) what steps in SQL are needed to convert - the old schema to the new one. - This is called a "migration". -4. Create a file ``migrations.py`` that will contain all - your migrations and add your new migration. +Luckily, Alembic can once again help with autogenerating what is +probably very close to the migration you want. First you will need to +find out what the revision id of your plugin's most recent migrations +is. There are two ways to do this: look in your plugin's migrations/ +directory and figure it out with the hope that it's "obvious" in some +way. The second path: let Alembic give that info for you. -Take a look at the core's ``db/migrations.py`` for some -good examples on what you might be able to do. Here's a -simple one to add one column: +Assuming you've already done the latest dbupdate with your plugin +enabled, do the following: -.. code-block:: python + ./bin/gmg alembic --with-plugins heads + +You should see the latest migration id for your plugin's label. - from mediagoblin.db.migration_tools import RegisterMigration, inspect_table - from sqlalchemy import MetaData, Column, Integer +Make changes to your +plugin's models.py and then run:: - MIGRATIONS = {} + ./bin/gmg alembic --with-plugins revision \ + --head REVISION_HERE \ + --autogenerate \ + -m "YOUR_PLUGIN_NAME: Change explanation here." - @RegisterMigration(1, MIGRATIONS) - def add_license_preference(db): - metadata = MetaData(bind=db.bind) +Once again, this will dump the migration into the wrong place, so move +to your plugin's migrations directory. Open the file, adjust +accordingly, and read carefully! Now you should also test your +migration with some real data. Be sure to test it both on SQLite +*AND* on PostgreSQL! - security_table = inspect_table(metadata, 'yourplugin__media_security') +One last *very critical* note: you must never, never modify core +tables with your plugin. To do that is to put you and all your users +in a dangerous situation. Add data to the database by adding new tables +under the control of your plugin, but never ever modify anyone else's! - col = Column('security_level', Integer) - col.create(security_table) - db.commit() +Whew, you made it! Get yourself a cookie to celebrate! diff --git a/docs/source/pluginwriter/hooks.rst b/docs/source/pluginwriter/hooks.rst new file mode 100644 index 00000000..66fe3b4d --- /dev/null +++ b/docs/source/pluginwriter/hooks.rst @@ -0,0 +1,35 @@ +.. MediaGoblin Documentation + + Written in 2014 by MediaGoblin contributors + + To the extent possible under law, the author(s) have dedicated all + copyright and related and neighboring rights to this software to + the public domain worldwide. This software is distributed without + any warranty. + + You should have received a copy of the CC0 Public Domain + Dedication along with this software. If not, see + <http://creativecommons.org/publicdomain/zero/1.0/>. + + +=============================== +Documentation on Built-in Hooks +=============================== + +This section explains built-in hooks to MediaGoblin. + + +What hooks are available? +========================= + +'collection_add_media' +---------------------- + +This hook is used by ``add_media_to_collection`` +in ``mediagoblin.user_pages.lib``. +It gets a ``CollectionItem`` as its argument. +It's the newly created item just before getting committed. +So the item can be modified by the hook, if needed. +Changing the session regarding this item is currently +undefined behaviour, as the SQL Session might contain other +things. diff --git a/docs/source/pluginwriter/quickstart.rst b/docs/source/pluginwriter/quickstart.rst index 6d45ea36..212c91ad 100644 --- a/docs/source/pluginwriter/quickstart.rst +++ b/docs/source/pluginwriter/quickstart.rst @@ -111,7 +111,7 @@ The code for ``__init__.py`` looks like this: :emphasize-lines: 12,23 import logging - from mediagoblin.tools.pluginapi import Plugin, get_config + from mediagoblin.tools.pluginapi import PluginManager, get_config # This creates a logger that you can use to log information to diff --git a/docs/source/pluginwriter/tests.rst b/docs/source/pluginwriter/tests.rst index fe99688f..560a8899 100644 --- a/docs/source/pluginwriter/tests.rst +++ b/docs/source/pluginwriter/tests.rst @@ -19,7 +19,7 @@ Here's a brief guide to writing unit tests for plugins. However, it isn't really ideal. It also hasn't been well tested... yes, there's some irony there :) -Some notes: we're using py.test and webtest for unit testing stuff. +Some notes: we're using py.test and WebTest for unit testing stuff. Keep that in mind. My suggestion is to mime the behavior of `mediagoblin/tests/` and put @@ -44,7 +44,7 @@ In any test module in your tests directory you can then do:: # real code goes here pass -And you'll get a mediagoblin application wrapped in webtest passed in +And you'll get a MediaGoblin application wrapped in WebTest passed in to your environment. If your plugin needs to define multiple configuration setups, you can @@ -52,8 +52,8 @@ actually set up multiple fixtures very easily for this. You can just set up multiple fixtures with different names that point to different configs and pass them in as that named argument. -To run the tests, from mediagoblin's directory (make sure that your -plugin has been added to your mediagoblin checkout's virtualenv!) do:: +To run the tests, from MediaGoblin's directory (make sure that your +plugin has been added to your MediaGoblin checkout's virtualenv!) do:: ./runtests.sh /path/to/myplugin/tests/ diff --git a/docs/source/siteadmin/commandline-upload.rst b/docs/source/siteadmin/commandline-upload.rst index 15b2377d..ef597a44 100644 --- a/docs/source/siteadmin/commandline-upload.rst +++ b/docs/source/siteadmin/commandline-upload.rst @@ -15,7 +15,13 @@ Command-line uploading ====================== -Want to submit media via the command line? It's fairly easy to do:: +If you're a site administrator and have access to the server then you +can use the 'addmedia' task. If you're just a user and want to upload +media by the command line you can. This can be done with the pump.io +API. There is `p <https://github.com/xray7224/p/>`_, which will allow you +to easily upload media from the command line, follow p's docs to do that. + +To use the addmedia command:: ./bin/gmg addmedia username your_media.jpg @@ -60,14 +66,14 @@ The above is an example of a very simple metadata.csv file. The batchaddmedia script would read this and attempt to upload only two pieces of media, and would be able to automatically name them appropriately. -The csv file +The CSV file ============ The location column ------------------- The location column is the one column that is absolutely necessary for uploading your media. This gives a path to each piece of media you upload. This can either a path to a local file or a direct link to remote media (with the -link in http format). As you can see in the example above the (fake) media was +link in HTTP format). As you can see in the example above the (fake) media was stored remotely on "www.example.net". Other internal nodes @@ -78,8 +84,8 @@ provide default information for your media entry, but as you'll see below, it's just as easy to provide this information through the correct metadata columns. - **id** is used to identify the media entry to the user in case of an error in the batchaddmedia script. -- **license** is used to set a license for your piece a media for mediagoblin's use. This must be a URI. -- **title** will set the title displayed to mediagoblin users. +- **license** is used to set a license for your piece a media for MediaGoblin's use. This must be a URI. +- **title** will set the title displayed to MediaGoblin users. - **description** will set a description of your media. Metadata columns @@ -101,7 +107,7 @@ information of uploaded media entries. - **dc:title** sets a title for your media entry. - **dc:description** sets a description of your media entry. -If both a metadata column and an internal node for the title are provided, mediagoblin +If both a metadata column and an internal node for the title are provided, MediaGoblin will use the internal node as the media entry's display name. This makes it so that if you want to display a piece of media with a different title than the one provided in its metadata, you can just provide different data for diff --git a/docs/source/siteadmin/configuration.rst b/docs/source/siteadmin/configuration.rst index 3da5cdd9..6b8cd225 100644 --- a/docs/source/siteadmin/configuration.rst +++ b/docs/source/siteadmin/configuration.rst @@ -39,12 +39,12 @@ paste.ini <http://pythonpaste.org/script/>`_). It also sets up some middleware that you can mostly ignore, except to configure sessions... more on that later. If you are adding a different - Python server other than fastcgi / plain HTTP, you might configure + Python server other than FastCGI / plain HTTP, you might configure it here. You probably won't need to change this file very much. There's one more file that you certainly won't change unless you're -making coding contributions to mediagoblin, but which can be useful to +making coding contributions to MediaGoblin, but which can be useful to read and reference: mediagoblin/config_spec.ini @@ -109,6 +109,34 @@ they sound like. - email_smtp_user - email_smtp_pass +Changing data directory +----------------------- + +MediaGoblin by default stores your data in wherever ``data_basedir``. +This can be changed by changing the value in your ``mediagoblin.ini`` file +for example:: + + [DEFAULT] + data_basedir = "/var/mediagoblin/user_data" + +For efficiency reasons MediaGoblin doesn't serve these files itself and +instead leaves that to the webserver. You will have to alter the location +to match the path in ``data_basedir``. + +If you use ``lazyserver.sh`` you need to change the ``paste.ini`` file:: + + [app:mediagoblin] + /mgoblin_media = /var/mediagoblin/user_data + +If you use Nginx you need to change the config:: + + # Instance specific media: + location /mgoblin_media/ { + alias /var/mediagoblin/user_data; + } + +Once you have done this you will need to move any existing media you had in the +old directory to the new directory so existing media still can be displayed. All other configuration changes ------------------------------- diff --git a/docs/source/siteadmin/deploying.rst b/docs/source/siteadmin/deploying.rst index 3f4a59cd..80b60642 100644 --- a/docs/source/siteadmin/deploying.rst +++ b/docs/source/siteadmin/deploying.rst @@ -17,25 +17,37 @@ Deploying MediaGoblin ===================== -GNU MediaGoblin is fairly new and so at the time of writing, there -aren't easy package-manager-friendly methods to install MediaGoblin. -However, doing a basic install isn't too complex in and of itself. +GNU MediaGoblin is fairly new, and so at the time of writing there aren't +easy package-manager-friendly methods to install it. However, doing a basic +install isn't too complex in and of itself. Following this deployment guide +will take you step-by-step through setting up your own instance of MediaGoblin. -There's an almost infinite way to deploy things... for now, we'll keep -it simple with some assumptions and use a setup that combines -mediagoblin + virtualenv + fastcgi + nginx on a .deb or .rpm based -GNU/Linux distro. +Of course, when it comes to setting up web applications like MediaGoblin, +there's an almost infinite way to deploy things, so for now, we'll keep it +simple with some assumptions. We recommend a setup that combines MediaGoblin + +virtualenv + FastCGI + Nginx on a .deb- or .rpm-based GNU/Linux distro. + +Other deployment options (e.g., deploying on FreeBSD, Arch Linux, using +Apache, etc.) are possible, though! If you'd prefer a different deployment +approach, see our +`Deployment wiki page <http://wiki.mediagoblin.org/Deployment>`_. .. note:: These tools are for site administrators wanting to deploy a fresh - install. If instead you want to join in as a contributor, see our + install. If you want to join in as a contributor, see our `Hacking HOWTO <http://wiki.mediagoblin.org/HackingHowto>`_ instead. - There are also many ways to install servers... for the sake of - simplicity, our instructions below describe installing with nginx. - For more recipes, including Apache, see - `our wiki <http://wiki.mediagoblin.org/Deployment>`_. +.. note:: + + Throughout the documentation we use the ``sudo`` command to indicate that + an instruction requires elevated user privileges to run. You can issue + these commands as the ``root`` user if you prefer. + + If you need help configuring ``sudo``, see the + `Debian wiki <https://wiki.debian.org/sudo/>`_ or the + `Fedora Project wiki <https://fedoraproject.org/wiki/Configuring_Sudo/>`_. + Prepare System -------------- @@ -45,25 +57,32 @@ Dependencies MediaGoblin has the following core dependencies: -- Python 2.6 or 2.7 +- Python 2.7 or Python 3.4+ - `python-lxml <http://lxml.de/>`_ - `git <http://git-scm.com/>`_ - `SQLite <http://www.sqlite.org/>`_/`PostgreSQL <http://www.postgresql.org/>`_ - `Python Imaging Library <http://www.pythonware.com/products/pil/>`_ (PIL) - `virtualenv <http://www.virtualenv.org/>`_ +- `nodejs <https://nodejs.org>`_ -On a DEB-based system (e.g Debian, gNewSense, Trisquel, Ubuntu, and +On a DEB-based system (e.g Debian, gNewSense, Trisquel, *buntu, and derivatives) issue the following command:: sudo apt-get install git-core python python-dev python-lxml \ - python-imaging python-virtualenv + python-imaging python-virtualenv npm nodejs-legacy automake \ + nginx On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the following command:: - yum install python-paste-deploy python-paste-script \ + sudo yum install python-paste-deploy python-paste-script \ git-core python python-devel python-lxml python-imaging \ - python-virtualenv + python-virtualenv npm automake nginx + +(Note: MediaGoblin now officially supports Python 3. You may instead +substitute from "python" to "python3" for most package names in the +Debian instructions and this should cover dependency installation. +These instructions have not yet been tested on Fedora.) Configure PostgreSQL ~~~~~~~~~~~~~~~~~~~~ @@ -75,31 +94,54 @@ Configure PostgreSQL For medium to large deployments we recommend PostgreSQL. - If you don't want/need postgres, skip this section. + If you don't want/need PostgreSQL, skip this section. -These are the packages needed for Debian Wheezy (stable):: +These are the packages needed for Debian Jessie (stable):: sudo apt-get install postgresql postgresql-client python-psycopg2 +These are the packages needed for an RPM-based system:: + + sudo yum install postgresql postgresql-server python-psycopg2 + +An rpm-based system also requires that you initialize and start the +PostgreSQL database with a few commands. The following commands are +not needed on a Debian-based platform, however:: + + sudo /usr/bin/postgresql-setup initdb + sudo systemctl enable postgresql + sudo systemctl start postgresql + The installation process will create a new *system* user named ``postgres``, -it will have privilegies sufficient to manage the database. We will create a -new database user with restricted privilegies and a new database owned by our +which will have privileges sufficient to manage the database. We will create a +new database user with restricted privileges and a new database owned by our restricted database user for our MediaGoblin instance. In this example, the database user will be ``mediagoblin`` and the database name will be ``mediagoblin`` too. -To create our new user, run:: +We'll add these entities by first switching to the *postgres* account:: + + sudo su - postgres - sudo -u postgres createuser -A -D mediagoblin +This will change your prompt to a shell prompt, such as *-bash-4.2$*. Enter +the following *createuser* and *createdb* commands at that prompt. We'll +create the *mediagoblin* database user first:: -then create the database all our MediaGoblin data should be stored in:: + # this command and the one that follows are run as the ``postgres`` user: + createuser -A -D mediagoblin - sudo -u postgres createdb -E UNICODE -O mediagoblin mediagoblin +Then we'll create the database where all of our MediaGoblin data will be stored:: + + createdb -E UNICODE -O mediagoblin mediagoblin where the first ``mediagoblin`` is the database owner and the second ``mediagoblin`` is the database name. +Type ``exit`` to exit from the 'postgres' user account.:: + + exit + .. caution:: Where is the password? These steps enable you to authenticate to the database in a password-less @@ -118,35 +160,37 @@ Drop Privileges for MediaGoblin MediaGoblin does not require special permissions or elevated access to run. As such, the preferred way to run MediaGoblin is to create a dedicated, unprivileged system user for the sole purpose of running -MediaGoblin. Running MediaGoblin processes under an unpriviledged system user +MediaGoblin. Running MediaGoblin processes under an unprivileged system user helps to keep it more secure. The following command (entered as root or with sudo) will create a system account with a username of ``mediagoblin``. You may choose a different -username if you wish.:: - - adduser --system mediagoblin +username if you wish. -No password will be assigned to this account, and you will not be able -to log in as this user. To switch to this account, enter either:: +If you are using a Debian-based system, enter this command:: - sudo -u mediagoblin /bin/bash # (if you have sudo permissions) + sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g www-data mediagoblin -or:: +If you are using an RPM-based system, enter this command:: - su mediagoblin -s /bin/bash # (if you have to use root permissions) + sudo useradd -c "GNU MediaGoblin system account" -d /var/lib/mediagoblin -m -r -g nginx mediagoblin -You may get a warning similar to this when entering these commands:: +This will create a ``mediagoblin`` user and assign it to a group that is +associated with the web server. This will ensure that the web server can +read the media files (images, videos, etc.) that users upload. - warning: cannot change directory to /home/mediagoblin: No such file or directory - -You can disregard this warning. To return to your regular user account after -using the system account, just enter ``exit``. +We will also create a ``mediagoblin`` group and associate the mediagoblin +user with that group, as well:: + + sudo groupadd mediagoblin && sudo usermod --append -G mediagoblin mediagoblin + +No password will be assigned to this account, and you will not be able +to log in as this user. To switch to this account, enter:: -.. note:: + sudo su mediagoblin -s /bin/bash - Unless otherwise noted, the remainder of this document assumes that all - operations are performed using this unpriviledged account. +To return to your regular user account after using the system account, type +``exit``. .. _create-mediagoblin-directory: @@ -156,73 +200,74 @@ Create a MediaGoblin Directory You should create a working directory for MediaGoblin. This document assumes your local git repository will be located at ``/srv/mediagoblin.example.org/mediagoblin/``. -Substitute your prefered local deployment path as needed. +Substitute your preferred local deployment path as needed. Setting up the working directory requires that we first create the directory -with elevated priviledges, and then assign ownership of the directory -to the unpriviledged system account. +with elevated privileges, and then assign ownership of the directory +to the unprivileged system account. -To do this, enter either of the following commands, changing the defaults -to suit your particular requirements:: +To do this, enter the following command, changing the defaults to suit your +particular requirements. On a Debian-based platform you will enter this:: - sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:mediagoblin /srv/mediagoblin.example.org + sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:www-data /srv/mediagoblin.example.org -or (as the root user):: +On an RPM-based distribution, enter this command:: - mkdir -p /srv/mediagoblin.example.org && chown -hR mediagoblin:mediagoblin /srv/mediagoblin.example.org + sudo mkdir -p /srv/mediagoblin.example.org && sudo chown -hR mediagoblin:nginx /srv/mediagoblin.example.org +.. note:: -Install MediaGoblin and Virtualenv ----------------------------------- + Unless otherwise noted, the remainder of this document assumes that all + operations are performed using this unprivileged account. -.. note:: - MediaGoblin is still developing rapidly. As a result - the following instructions recommend installing from the ``master`` - branch of the git repository. Eventually production deployments will - want to transition to running from more consistent releases. +Install MediaGoblin and Virtualenv +---------------------------------- -We will now clone the MediaGoblin source code repository and setup and -configure the necessary services. Modify these commands to -suit your own environment. As a reminder, you should enter these -commands using your unpriviledged system account. +We will now switch to our 'mediagoblin' system account, and then set up +our MediaGoblin source code repository and its necessary services. +You should modify these commands to suit your own environment. Change to the MediaGoblin directory that you just created:: - cd /srv/mediagoblin.example.org + sudo su mediagoblin -s /bin/bash # to change to the 'mediagoblin' account + $ cd /srv/mediagoblin.example.org Clone the MediaGoblin repository and set up the git submodules:: - git clone git://gitorious.org/mediagoblin/mediagoblin.git - cd mediagoblin - git submodule init && git submodule update + $ git clone git://git.savannah.gnu.org/mediagoblin.git -b stable + $ cd mediagoblin + $ git submodule init && git submodule update +.. note:: -And set up the in-package virtualenv:: + The MediaGoblin repository used to be on gitorious.org, but since + gitorious.org shut down, we had to move. We are presently on + Savannah. You may need to update your git repository location:: - (virtualenv --system-site-packages . || virtualenv .) && ./bin/python setup.py develop + $ git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git -.. note:: +Set up the hacking environment:: + + $ ./bootstrap.sh && ./configure && make - We presently have an **experimental** make-style deployment system. if - you'd like to try it, instead of the above command, you can run:: +(Note that if you'd prefer to run MediaGoblin with Python 3, pass in +`--with-python3` to the `./configure` command.) - ./experimental-bootstrap.sh && ./configure && make +Create and set the proper permissions on the ``user_dev`` directory. +This directory will be used to store uploaded media files:: - This also includes a number of nice features, such as keeping your - viratualenv up to date by simply running `make update`. + $ mkdir user_dev && chmod 750 user_dev - Note: this is liable to break. Use this method with caution. +Assuming you are going to deploy with FastCGI, you should also install +flup:: -.. :: + $ ./bin/easy_install flup - (NOTE: Is this still relevant?) +(Note, if you're running Python 2, which you probably are at this +point in MediaGoblin's development, you'll need to run:) - If you have problems here, consider trying to install virtualenv - with the ``--distribute`` or ``--no-site-packages`` options. If - your system's default Python is in the 3.x series you may need to - run ``virtualenv`` with the ``--python=python2.7`` or - ``--python=python2.6`` options. + $ ./bin/easy_install flup==1.0.3.dev-20110405 The above provides an in-package install of ``virtualenv``. While this is counter to the conventional ``virtualenv`` configuration, it is @@ -230,26 +275,36 @@ more reliable and considerably easier to configure and illustrate. If you're familiar with Python packaging you may consider deploying with your preferred method. -Assuming you are going to deploy with FastCGI, you should also install -flup:: - - ./bin/easy_install flup - -(Sometimes this breaks because flup's site is flakey. If it does for -you, try):: - - ./bin/easy_install https://pypi.python.org/pypi/flup/1.0.3.dev-20110405 +.. note:: -This concludes the initial configuration of the development + What if you don't want an in-package ``virtualenv``? Maybe you + have your own ``virtualenv``, or you are building a MediaGoblin + package for a distribution. There's no need necessarily for the + virtualenv produced by ``./configure && make`` by default other + than attempting to simplify work for developers and people + deploying by hiding all the virtualenv and bower complexity. + + If you want to install all of MediaGoblin's libraries + independently, that's totally fine! You can pass the flag + ``--without-virtualenv`` which will skip this step. + But you will need to install all those libraries manually and make + sure they are on your ``PYTHONPATH`` yourself! (You can still use + ``python setup.py develop`` to install some of those libraries, + but note that no ``./bin/python`` will be set up for you via this + method, since no virtualenv is set up for you!) + +This concludes the initial configuration of the MediaGoblin environment. In the future, when you update your codebase, you should also run:: - git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate + $ git submodule update && ./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate -Note: If you are running an active site, depending on your server -configuration, you may need to stop it first or the dbupdate command -may hang (and it's certainly a good idea to restart it after the -update) +.. note:: + + Note: If you are running an active site, depending on your server + configuration, you may need to stop it first or the dbupdate command + may hang (and it's certainly a good idea to restart it after the + update) Deploy MediaGoblin Services @@ -259,23 +314,24 @@ Edit site configuration ~~~~~~~~~~~~~~~~~~~~~~~ A few basic properties must be set before MediaGoblin will work. First -make a copy of ``mediagoblin.ini`` for editing so the original config -file isn't lost:: +make a copy of ``mediagoblin.ini`` and ``paste.ini`` for editing so the original +config files aren't lost (you likely won't need to edit the paste configuration, +but we'll make a local copy of it just in case):: - cp mediagoblin.ini mediagoblin_local.ini + $ cp -av mediagoblin.ini mediagoblin_local.ini && cp -av paste.ini paste_local.ini -Then: +Then edit mediagoblin_local.ini: - Set ``email_sender_address`` to the address you wish to be used as the sender for system-generated emails - Edit ``direct_remote_path``, ``base_dir``, and ``base_url`` if your mediagoblin directory is not the root directory of your - vhost. + site. Configure MediaGoblin to use the PostgreSQL database ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If you are using postgres, edit the ``[mediagoblin]`` section in your +If you are using PostgreSQL, edit the ``[mediagoblin]`` section in your ``mediagoblin_local.ini`` and put in:: sql_engine = postgresql:///mediagoblin @@ -289,7 +345,7 @@ Update database data structures Before you start using the database, you need to run:: - ./bin/gmg dbupdate + $ ./bin/gmg dbupdate to populate the database with the MediaGoblin data structures. @@ -300,20 +356,25 @@ Test the Server At this point MediaGoblin should be properly installed. You can test the deployment with the following command:: - ./lazyserver.sh --server-name=broadcast + $ ./lazyserver.sh --server-name=broadcast You should be able to connect to the machine on port 6543 in your browser to confirm that the service is operable. +The next series of commands will need to be run as a privileged user. Type +exit to return to the root/sudo account.:: + + exit + .. _webserver-config: FastCGI and nginx ~~~~~~~~~~~~~~~~~ -This configuration example will use nginx, however, you may +This configuration example will use Nginx, however, you may use any webserver of your choice as long as it supports the FastCGI -protocol. If you do not already have a web server, consider nginx, as +protocol. If you do not already have a web server, consider Nginx, as the configuration files may be more clear than the alternatives. @@ -321,13 +382,22 @@ Create a configuration file at ``/srv/mediagoblin.example.org/nginx.conf`` and create a symbolic link into a directory that will be included in your ``nginx`` configuration (e.g. "``/etc/nginx/sites-enabled`` or ``/etc/nginx/conf.d``) with -one of the following commands (as the root user):: +one of the following commands. - ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/ - ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/ +On a DEB-based system (e.g Debian, gNewSense, Trisquel, *buntu, and +derivatives) issue the following commands:: + + sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/ + sudo systemctl enable nginx + +On a RPM-based system (e.g. Fedora, RedHat, and derivatives) issue the +following commands:: -Modify these commands and locations depending on your preferences and -the existing configuration of your nginx instance. The contents of + sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/ + sudo systemctl enable nginx + +You can modify these commands and locations depending on your preferences and +the existing configuration of your Nginx instance. The contents of this ``nginx.conf`` file should be modeled on the following:: server { @@ -344,7 +414,7 @@ this ``nginx.conf`` file should be modeled on the following:: gzip on; gzip_min_length 1024; gzip_buffers 4 32k; - gzip_types text/plain text/html application/x-javascript text/javascript text/xml text/css; + gzip_types text/plain application/x-javascript text/javascript text/xml text/css; ##################################### # Mounting MediaGoblin stuff @@ -387,25 +457,43 @@ this ``nginx.conf`` file should be modeled on the following:: fastcgi_pass 127.0.0.1:26543; include /etc/nginx/fastcgi_params; - # our understanding vs nginx's handling of script_name vs + # our understanding vs Nginx's handling of script_name vs # path_info don't match :) fastcgi_param PATH_INFO $fastcgi_script_name; fastcgi_param SCRIPT_NAME ""; } } -Now, nginx instance is configured to serve the MediaGoblin -application. Perform a quick test to ensure that this configuration -works. Restart nginx so it picks up your changes, with a command that -resembles one of the following (as the root user):: +The first four ``location`` directives instruct Nginx to serve the +static and uploaded files directly rather than through the MediaGoblin +process. This approach is faster and requires less memory. + +.. note:: + + The user who owns the Nginx process, normally ``www-data`` or ``nginx``, + requires execute permission on the directories ``static``, + ``public``, ``theme_static`` and ``plugin_static`` plus all their + parent directories. This user also requires read permission on all + the files within these directories. This is normally the default. + +Nginx is now configured to serve the MediaGoblin application. Perform a quick +test to ensure that this configuration works:: + + nginx -t + +If you encounter any errors, review your Nginx configuration files, and try to +resolve them. If you do not encounter any errors, you can start your Nginx +server with one of the following commands (depending on your environment):: sudo /etc/init.d/nginx restart sudo /etc/rc.d/nginx restart + sudo systemctl restart nginx Now start MediaGoblin. Use the following command sequence as an example:: cd /srv/mediagoblin.example.org/mediagoblin/ + su mediagoblin -s /bin/bash ./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543 Visit the site you've set up in your browser by visiting @@ -426,6 +514,29 @@ Instructions and scripts for running MediaGoblin on an Apache server can be found on the `MediaGoblin wiki <http://wiki.mediagoblin.org/Deployment>`_. +Should I Keep Open Registration Enabled? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Unfortunately, in this current release of MediaGoblin we are suffering +from spammers registering to public instances en masse. As such, you +may want to either: + +a) Disable registration on your instance and just make + accounts for people you know and trust (eg via the `gmg adduser` + command). You can disable registration in your mediagoblin.ini + like so:: + + [mediagoblin] + allow_registration = false + +b) Enable a CAPTCHA plugin. But unfortunately, though some CAPTCHA + plugins exist, for various reasons we do not have any general + recommendations we can make at this point. + +We hope to have a better solution to this situation shortly. We +apologize for the inconvenience in the meanwhile. + + Security Considerations ~~~~~~~~~~~~~~~~~~~~~~~ @@ -439,3 +550,7 @@ Security Considerations and restart the server, so that it creates a new secret key. All previous sessions will be invalidated. +.. + Local variables: + fill-column: 70 + End: diff --git a/docs/source/siteadmin/foreword.rst b/docs/source/siteadmin/foreword.rst index 4c425f8d..24282f96 100644 --- a/docs/source/siteadmin/foreword.rst +++ b/docs/source/siteadmin/foreword.rst @@ -34,13 +34,14 @@ Improving the Site Administrator's Guide There are a few ways---please pick whichever method is convenient for you! -1. Write up a bug report in the bug tracker +1. Write up a bug report in the `bug tracker`_. 2. Tell someone on IRC ``#mediagoblin`` on Freenode. -3. Write an email to the devel mailing list. +3. Write an email to the `devel mailing list`_. -Information about the bugtracker, IRC and the mailing list is all on -the `join page`_. +More information about contributing is available on the `join page`_. +.. _bug tracker: https://issues.mediagoblin.org/ +.. _devel mailing list: http://lists.mediagoblin.org/listinfo/devel .. _join page: http://mediagoblin.org/join/ Patches are the most helpful, but even feedback on what you think diff --git a/docs/source/siteadmin/media-types.rst b/docs/source/siteadmin/media-types.rst index 3e8a94e9..146e1aae 100644 --- a/docs/source/siteadmin/media-types.rst +++ b/docs/source/siteadmin/media-types.rst @@ -1,6 +1,6 @@ .. MediaGoblin Documentation - Written in 2011, 2012 by MediaGoblin contributors + Written in 2011, 2012, 2014, 2015 by MediaGoblin contributors To the extent possible under law, the author(s) have dedicated all copyright and related and neighboring rights to this software to @@ -18,11 +18,11 @@ Media Types ==================== In the future, there will be all sorts of media types you can enable, -but in the meanwhile there are five additional media types: video, audio, -ascii art, STL/3d models, PDF and Document. +but in the meanwhile there are six additional media types: video, audio, +raw image, ASCII art, STL/3D models, PDF and Document. First, you should probably read ":doc:`configuration`" to make sure -you know how to modify the mediagoblin config file. +you know how to modify the MediaGoblin config file. Enabling Media Types ==================== @@ -30,7 +30,7 @@ Enabling Media Types .. note:: Media types are now plugins -Media types are enabled in your mediagoblin configuration file, typically it is +Media types are enabled in your MediaGoblin configuration file, typically it is created by copying ``mediagoblin.ini`` to ``mediagoblin_local.ini`` and then applying your changes to ``mediagoblin_local.ini``. If you don't already have a ``mediagoblin_local.ini``, create one in the way described. @@ -69,22 +69,33 @@ The file-extension-based approach is used before the sniffing-based approach, if the file-extension-based approach finds a match, the sniffing-based approach will be skipped as it uses far more processing power. +Configuring Media Types +======================= + +Each media type has a ``config_spec.ini`` file with configurable +options and comments explaining their intended side effect. For +instance the ``video`` media type configuration can be found in +``mediagoblin/media_types/video/config_spec.ini``. + Video ===== -To enable video, first install gstreamer and the python-gstreamer -bindings (as well as whatever gstremaer extensions you want, +To enable video, first install GStreamer and the python-gstreamer +bindings (as well as whatever GStreamer extensions you want, good/bad/ugly). On Debianoid systems .. code-block:: bash - sudo apt-get install python-gst0.10 \ - gstreamer0.10-plugins-base \ - gstreamer0.10-plugins-bad \ - gstreamer0.10-plugins-good \ - gstreamer0.10-plugins-ugly \ - gstreamer0.10-ffmpeg + sudo apt-get install python-gi python3-gi \ + gstreamer1.0-tools \ + gir1.2-gstreamer-1.0 \ + gir1.2-gst-plugins-base-1.0 \ + gstreamer1.0-plugins-good \ + gstreamer1.0-plugins-ugly \ + gstreamer1.0-plugins-bad \ + gstreamer1.0-libav \ + python-gst-1.0 Add ``[[mediagoblin.media_types.video]]`` under the ``[plugins]`` section in @@ -96,7 +107,7 @@ Run ./bin/gmg dbupdate -Now you should be able to submit videos, and mediagoblin should +Now you should be able to submit videos, and MediaGoblin should transcode them. .. note:: @@ -110,28 +121,18 @@ transcode them. Audio ===== -To enable audio, install the gstreamer and python-gstreamer bindings (as well -as whatever gstreamer plugins you want, good/bad/ugly), scipy and numpy are +To enable audio, install the GStreamer and python-gstreamer bindings (as well +as whatever GStreamer plugins you want, good/bad/ugly), SciPy and NumPy are also needed for the audio spectrograms. To install these on Debianoid systems, run:: - sudo apt-get install python-gst0.10 gstreamer0.10-plugins-{base,bad,good,ugly} \ - gstreamer0.10-ffmpeg python-numpy python-scipy - -The ``scikits.audiolab`` package you will install in the next step depends on the -``libsndfile1-dev`` package, so we should install it. -On Debianoid systems, run - -.. code-block:: bash - - sudo apt-get install libsndfile1-dev + sudo apt-get install python-gst-1.0 gstreamer1.0-plugins-{base,bad,good,ugly} \ + gstreamer1.0-libav python-numpy python-scipy libsndfile1-dev libasound2-dev .. note:: scikits.audiolab will display a warning every time it's imported if you do not compile it with alsa support. Alsa support is not necessary for the GNU - MediaGoblin application but if you do not wish the alsa warnings from - audiolab you should also install ``libasound2-dev`` before installing - scikits.audiolab. + MediaGoblin application. Then install ``scikits.audiolab`` for the spectrograms:: @@ -149,12 +150,34 @@ Run You should now be able to upload and listen to audio files! -Ascii art +Raw image ========= -To enable ascii art support, first install the +To enable raw image you need to install pyexiv2. On Debianoid systems + +.. code-block:: bash + + sudo apt-get install python-pyexiv2 + +Add ``[[mediagoblin.media_types.raw_image]]`` under the ``[plugins]`` +section in your ``mediagoblin_local.ini`` and restart MediaGoblin. + +Run + +.. code-block:: bash + + ./bin/gmg dbupdate + +Now you should be able to submit raw images, and MediaGoblin should +extract the JPEG preview from them. + + +ASCII art +========= + +To enable ASCII art support, first install the `chardet <http://pypi.python.org/pypi/chardet>`_ -library, which is necessary for creating thumbnails of ascii art +library, which is necessary for creating thumbnails of ASCII art .. code-block:: bash @@ -171,20 +194,20 @@ Run ./bin/gmg dbupdate -Now any .txt file you uploaded will be processed as ascii art! +Now any .txt file you uploaded will be processed as ASCII art! -STL / 3d model support +STL / 3D model support ====================== -To enable the "STL" 3d model support plugin, first make sure you have -a recentish `Blender <http://blender.org>`_ installed and available on +To enable the "STL" 3D model support plugin, first make sure you have +a recent `Blender <http://blender.org>`_ installed and available on your execution path. This feature has been tested with Blender 2.63. It may work on some earlier versions, but that is not guaranteed (and is surely not to work prior to Blender 2.5X). Add ``[[mediagoblin.media_types.stl]]`` under the ``[plugins]`` section in your -``mediagoblin_local.ini`` and restart MediaGoblin. +``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -200,9 +223,9 @@ PDF and Document To enable the "PDF and Document" support plugin, you need: -1. pdftocairo and pdfinfo for pdf only support. +1. pdftocairo and pdfinfo for PDF only support. -2. unoconv with headless support to support converting libreoffice supported +2. unoconv with headless support to support converting LibreOffice supported documents as well, such as doc/ppt/xls/odf/odg/odp and more. For the full list see mediagoblin/media_types/pdf/processing.py, unoconv_supported. @@ -215,7 +238,7 @@ To install this on Fedora: sudo yum install -y poppler-utils unoconv libreoffice-headless -Note: You can leave out unoconv and libreoffice-headless if you want only pdf +Note: You can leave out unoconv and libreoffice-headless if you want only PDF support. This will result in a much smaller list of dependencies. pdf.js relies on git submodules, so be sure you have fetched them: @@ -233,7 +256,7 @@ This feature has been tested on Fedora with: It may work on some earlier versions, but that is not guaranteed. Add ``[[mediagoblin.media_types.pdf]]`` under the ``[plugins]`` section in your -``mediagoblin_local.ini`` and restart MediaGoblin. +``mediagoblin_local.ini`` and restart MediaGoblin. Run @@ -242,3 +265,13 @@ Run ./bin/gmg dbupdate +Blog (HIGHLY EXPERIMENTAL) +========================== + +MediaGoblin has a blog media type, which you might notice by looking +through the docs! However, it is *highly experimental*. We have not +security reviewed this, and it acts in a way that is not like normal +blogs (the blog posts are themselves media types!). + +So you can play with this, but it is not necessarily recommended yet +for production use! :) diff --git a/docs/source/siteadmin/plugins.rst b/docs/source/siteadmin/plugins.rst index baca381d..8682b0c7 100644 --- a/docs/source/siteadmin/plugins.rst +++ b/docs/source/siteadmin/plugins.rst @@ -55,7 +55,7 @@ offer for your media), we would do:: virtual environment before installing with pip. Otherwise the plugin may get installed in a different environment than the one MediaGoblin is installed in. Also make sure, you use e.g. pip-2.7 if your default - python (and thus pip) is python 3 (e.g. in Ubuntu). + python (and thus pip) is python 3 (e.g. in *buntu). Once you've installed the plugin software, you need to tell MediaGoblin that this is a plugin you want MediaGoblin to use. To do @@ -110,11 +110,32 @@ comments making the bits clearer):: Check the plugin's documentation for what configuration options are available. +Once you've set up your plugin, you should be sure to update the +database to accommodate the new plugins:: -Removing plugins -================ + ./bin/gmg dbupdate -To remove a plugin, use ``pip uninstall``. For example:: + +Deactivating plugins +==================== + +You should be aware that once you enable a plugin, deactivating it +might be a bit tricky, for migrations reasons. In the future we may +produce better tooling to accommodate this. In short, you will need to +do a bit of database surgery by: + +- Removing all tables and indexes installed by the plugin +- Removing the plugin's migration head id from the `alembic_version` + table. (You might be able to determine which to remove via + examining the output of `./bin/gmg alembic heads`) + +Note that this is a VERY TRICKY process, and you should be sure to make +a backup first. You've been warned! + +Removing plugin packages +======================== + +To remove an external plugin's package, use ``pip uninstall``. For example:: pip uninstall mediagoblin-licenses diff --git a/docs/source/siteadmin/production-deployments.rst b/docs/source/siteadmin/production-deployments.rst index 839d3ce5..ee915573 100644 --- a/docs/source/siteadmin/production-deployments.rst +++ b/docs/source/siteadmin/production-deployments.rst @@ -1,6 +1,6 @@ .. MediaGoblin Documentation - Written in 2011, 2012 by MediaGoblin contributors + Written in 2011, 2012, 2013, 2014, 2015 by MediaGoblin contributors To the extent possible under law, the author(s) have dedicated all copyright and related and neighboring rights to this software to @@ -19,15 +19,145 @@ This document contains a number of suggestions for deploying MediaGoblin in actual production environments. Consider ":doc:`deploying`" for a basic overview of how to deploy MediaGoblin. -Deploy with Paste +Deploy with paste ----------------- The MediaGoblin WSGI application instance you get with ``./lazyserver.sh`` is not ideal for a production MediaGoblin deployment. Ideally, you should be able -to use an "init" or "control" script to launch and restart the MediaGoblin -process. +to use a Systemd service file or an init script to launch and restart the +MediaGoblin process. -Use the following command as the basis for such a script: +We will explore setting up MediaGoblin Systemd service files and init scripts, +but first we need to create the directory that will store the MediaGoblin logs. + + +.. _create-log-file-dir: + +Create the directory for your log file: +--------------------------------------- + +Production logs for the MediaGoblin application are kept in the +``/var/log/mediagoblin`` directory. Create the directory and give it the +proper permissions:: + + sudo mkdir -p /var/log/mediagoblin && sudo chown -hR mediagoblin:mediagoblin /var/log/mediagoblin + + +.. _systemd-service-files: + +Use Systemd service files +------------------------- + +If your operating system uses Systemd, you can use Systemd ``service files`` +to manage both the Celery and Paste processes. Place the following service +files in the ``/etc/systemd/system/`` directory. + +The first file should be named ``mediagoblin-celeryd.service``. Be sure to +modify it to suit your environment's setup: + +.. code-block:: bash + + # Set the WorkingDirectory, Environment and ExecStart values to match your environment. + # If using Debian/*buntu, mkdir and chown are located in /bin/mkdir and /bin/chown, respectively. + # If using Fedora/CentOS/Red Hat, mkdir and chown are located in /usr/bin/mkdir and /usr/bin/chown, respectively. + + [Unit] + Description=MediaGoblin Celeryd + + [Service] + User=mediagoblin + Group=mediagoblin + Type=simple + WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin + # Start mg-celeryd process as root, then switch to mediagoblin user/group + # (This is needed to run the ExecStartPre commands) + PermissionsStartOnly=true + # Create directory for PID (if needed) and set ownership + ExecStartPre=/bin/mkdir -p /run/mediagoblin + ExecStartPre=/bin/chown -hR mediagoblin:mediagoblin /run/mediagoblin + # Celery process will run as the `mediagoblin` user after start. + Environment=MEDIAGOBLIN_CONFIG=/srv/mediagoblin.example.org/mediagoblin/mediagoblin_local.ini \ + CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery + ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/celery worker \ + --logfile=/var/log/mediagoblin/celery.log \ + --loglevel=INFO + PIDFile=/run/mediagoblin/mediagoblin-celeryd.pid + + [Install] + WantedBy=multi-user.target + + +The second file should be named ``mediagoblin-paster.service``: + + +.. code-block:: bash + + # Set the WorkingDirectory, Environment and ExecStart values to match your environment. + # If using Debian/*buntu, mkdir and chown are located in /bin/mkdir and /bin/chown, respectively. + # If using Fedora/CentOS/Red Hat, mkdir and chown are located in /usr/bin/mkdir and /usr/bin/chown, respectively. + [Unit] + Description=Mediagoblin + + [Service] + Type=forking + User=mediagoblin + Group=mediagoblin + Environment=CELERY_ALWAYS_EAGER=false + WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin + # Start mg-paster process as root, then switch to mediagoblin user/group + PermissionsStartOnly=true + ExecStartPre=-/bin/mkdir -p /run/mediagoblin + ExecStartPre=/bin/chown -hR mediagoblin:mediagoblin /run/mediagoblin + + ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ + /srv/mediagoblin.example.org/mediagoblin/paste_local.ini \ + --pid-file=/var/run/mediagoblin/mediagoblin.pid \ + --log-file=/var/log/mediagoblin/mediagoblin.log \ + --daemon \ + --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543 + ExecStop=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \ + --pid-file=/var/run/mediagoblin/mediagoblin.pid \ + /srv/mediagoblin.example.org/mediagoblin/paste_local.ini stop + PIDFile=/var/run/mediagoblin/mediagoblin.pid + + [Install] + WantedBy=multi-user.target + + + +Enable these processes to start at boot by entering:: + + sudo systemctl enable mediagoblin-celeryd.service && sudo systemctl enable mediagoblin-paster.service + + +Start the processes for the current session with:: + + sudo systemctl start mediagoblin-paster.service + sudo systemctl start mediagoblin-celeryd.service + + +If either command above gives you an error, you can investigate the cause of +the error by entering:: + + sudo systemctl status mediagoblin-celeryd.service or + sudo systemctl status mediagoblin-paster.service + +The above ``systemctl status`` command is also useful if you ever want to +confirm that a process is still running. If you make any changes to the service +files, you can reload the service files by entering:: + + sudo systemctl daemon-reload + +After entering that command, you can attempt to start the Celery or Paste +processes again. + +.. _init-script: + +Use an init script +------------------ + +If your system does not use Systemd, you can use the following command as the +basis for an init script: .. code-block:: bash @@ -53,7 +183,30 @@ as the basis for your script: --pid-file=/var/run/mediagoblin.pid \ --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543 -Separate Celery + +Members of the MediaGoblin community have provided init scripts for the +following GNU/Linux distributions: + +Debian + * `GNU MediaGoblin init scripts + <https://github.com/joar/mediagoblin-init-scripts>`_ + by `Joar Wandborg <http://wandborg.se>`_ + +Arch Linux + * `MediaGoblin - ArchLinux rc.d scripts + <http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts>`_ + by `Jeremy Pope <http://jpope.org/>`_ + * `Mediagoblin init script on Archlinux + <http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/>`_ + by `Chimo <http://chimo.chromic.org/>`_ + +You can reference these scripts to create an init script for your own operating +system. Similar scripts will be in your system's ``/etc/init.d/`` +or ``/etc/rc.d/`` directory, but the specifics of an init script will vary from +one distribution to the next. + + +Separate celery --------------- MediaGoblin uses `Celery`_ to handle heavy and long-running tasks. Celery can @@ -91,6 +244,11 @@ To launch Celery separately from the MediaGoblin WSGI application: CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd +If you use our example Systemd ``service files``, Celery will be set to the +"CELERY_ALWAYS_EAGER=false" value by default. This will provide your users +with the best user experience, as all media processing will be done in the +background. + .. _sentry: Set up sentry to monitor exceptions @@ -102,31 +260,6 @@ documentation. .. _`raven`: http://raven.readthedocs.org -.. _init-script: - -Use an Init Script ------------------- - -Look in your system's ``/etc/init.d/`` or ``/etc/rc.d/`` directory for -examples of how to build scripts that will start, stop, and restart -MediaGoblin and Celery. These scripts will vary by -distribution/operating system. - -These are scripts provided by the MediaGoblin community: - -Debian - * `GNU MediaGoblin init scripts - <https://github.com/joar/mediagoblin-init-scripts>`_ - by `Joar Wandborg <http://wandborg.se>`_ - -Arch Linux - * `MediaGoblin - ArchLinux rc.d scripts - <http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts>`_ - by `Jeremy Pope <http://jpope.org/>`_ - * `Mediagoblin init script on Archlinux - <http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/>`_ - by `Chimo <http://chimo.chromic.org/>`_ - .. TODO insert init script here .. TODO are additional concerns ? .. Other Concerns diff --git a/docs/source/siteadmin/relnotes.rst b/docs/source/siteadmin/relnotes.rst index 3542bdcb..f32ca792 100644 --- a/docs/source/siteadmin/relnotes.rst +++ b/docs/source/siteadmin/relnotes.rst @@ -21,6 +21,304 @@ This chapter has important information for releases in it. If you're upgrading from a previous release, please read it carefully, or at least skim over it. +.. note:: + + ALWAYS do backups before upgrading, especially before + running migrations! That way if something goes wrong, we can fix + things! + + And be sure to shut down your current MediaGoblin/Celery processes + before upgrading! + +.. note:: + + The MediaGoblin repository used to be on gitorious.org, but since + gitorious.org shut down, we had to move. We are presently on + Savannah. You may need to update your git repository location:: + + git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git + + +0.9.0 +===== + +This release has a number of improvements, but is also a major +"plumbing upgrade" release to MediaGoblin. Notably, we now support +Python 3, which is pretty cool! + +**Do this to upgrade** + +0. If you haven't already, switch the git remote URL: + ``git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git`` +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.9.0`` +2. Run + ``./bootstrap.sh && ./configure && make`` +3. Also run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +**Bugfixes/improvements:** + +- Python 3 is now a first class citizen! We now support both + Python 2.7 and Python 3.4 or later. +- Major updates to internal tooling to pave the way for federation. + - Massive overhaul to the database layout (particularly in + permitting generic relations) + - OAuth updates + - Updating how we handle collections + - Add a "graveyard" system with tombstones for keeping information + about removed objects + - Large overhaul to how "comments" work. In federation, many things + can reply to many things, so we had to loosen the model. +- If your user has some collections available, these will be presented + as a dropdown option while submitting media. +- Begin using Alembic for migrations +- Lots of bugfixes and etc + - Many fixes to typos + - Some fixes to the blog system + - Switch to waitress for development + - And more...! + + +0.8.1 +===== + +This release is a security and bugfix release. We recommend you upgrade as +soon as possible. + +**Do this to upgrade** + +0. If you haven't already, switch the git remote URL: + ``git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git`` +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.8.1`` +2. Run + ``./bootstrap.sh && ./configure && make`` +3. Also run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +(Please check intermediate release steps as well if not upgrading from +0.8.0) + +**Bugfixes/improvements:** + +Most importantly, there is an **important security fix**: + +Quoting here a portion of the +`release blogpost <http://mediagoblin.org/news/mediagoblin-0.8.1-security-release.html>`_:: + + We have had a security problem in our OAuth implementation reported to + us privately and have taken steps to address it. The security problem + affects all versions of GNU MediaGoblin since 0.5.0. I have created a patch + for this and released a minor version 0.8.1. It's strongly advised + that everyone upgrade as soon as they can. + + In order to exploit the security issue, an attacker must have had + access to a logged in session to your GNU MediaGoblin account. If you + have kept your username and password secret, logging in only over + HTTPS and you've not left yourself logged in on publicly accessible + computers, you should be safe. However it's still advised all users + take the following precautions, listed below. + + Users should check their authorized clients. Any client which looks + unfamiliar to you, you should deauthorize. To check this: + + 1) Log in to the GNU MediaGoblin instance + 2) Click the drop down arrow in the upper right + 3) Click "Change account settings" + 4) At the bottom click the "Deauthorize applications" link + + If you are unsure of any of these, click "Deauthorize". + +There are other bugfixes, but they are fairly minor. + + +0.8.0 +===== + +This release has a number of changes related to the way we recommend +building MediaGoblin; upgrade steps are below, but if you run into +trouble, consider pinging the MediaGoblin list or IRC channel. + +**Do this to upgrade** + +0. If you haven't already, switch the git remote URL: + ``git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git`` +1. If you don't have node.js installed, you'll need it for handling + MediaGoblin's static web dependencies. Install this via your + distribution! (In the glorious future MediaGoblin will be simply + packaged for your distribution so you won't have to worry about + this!) +2. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.8.0`` +3. Run + ``./bootstrap.sh && ./configure && make`` +4. Also run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +Please note the important steps of 0 and 2, which have not appeared in +prior upgrade guides! + +Additionally: + +- Are you using audio or video media types? In that case, you'll need + to update your GStreamer instance to 1.0. +- The Pump API needs some data passed through to the WSGI application, + so if you are using Apache with mod_wsgi you should be sure to make + sure to add "WSGIPassAuthorization On" to your config. (Using the + default MediaGoblin documentation and config, things should work + as-is.) + + +**Bugfixes/improvements:** + +- Preliminary / experimental support for Python 3! +- Footer forced to the bottom of page +- Massive improvements to Pump API support + - Able to run on multiple existing Pump clients! Including Pumpa + and Dianara! +- much cleaner ./configure && make support; it's now the default +- Clearer documentation on permissions and installation +- Switched from Transifex, which had become proprietary, to an + instance of Pootle hosted for GNU +- Moved to GStreamer 1.0! This also adds a new thumbnailer which + gives much better results in +- Removed terrible check-JavaScript-dependencies-into-your-application + setup, now using Bower for dependency tracking +- Put some scaffolding in place for Alembic, which will be used for + future migration work +- Automatically create a fresh mediagoblin.ini from + mediagoblin.ini.example +- no more need for mediagoblin_local.ini (though it's still supported) +- Fix lowercasing of username in auth steps +- Slowly moving towards removing global state (a source of many bugs) + +0.7.1 +===== + +This is a purely bugfix release. Important changes happened since +0.7.0; if running MediaGoblin 0.7.0, an upgrade is highly recommended; +see below. This release is especially useful if you have been running +PostgreSQL and have been receiving seemingly random database transaction +errors. + +**Do this to upgrade** + +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.7.1 && git submodule init && git submodule update`` +2. Make sure to run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +That's it, probably! If you run into problems, don't hesitate to +`contact us <http://mediagoblin.org/pages/join.html>`_ +(IRC is often best). + +**Bugfixes/improvements:** + +- The *MOST IMPORTANT* change in this release: + Disabling a couple of non-critical features that were causing + database transaction issues. (These should be back by 0.8.0.) + + + Disabled the "checking if the database is up to date at + MediaGoblin startup" feature + + Disabled the garbage collection stuff by default for now + (You can set garbage_collection under the config MediaGoblin + header to something other than 0 to turn it back on for now, but + it's potentially risky for the moment.) + +- Some fixes to the 0.7.0 docs +- Fixed Sandy 70s speedboat navbar by updating git submodule +- Added support for cr2 files in raw_image media type +- Added a description to setup.py +- Collection and CollectionItem objects now have nicer in-python representations +- Fixed Unicode error with raw image mediatype logging +- Fixed #945 "Host metadata does not confirm to spec (/.well-known/meta-data)" + + + Add XRD+XML formatting for /.well-known/host-meta + + Add /.well-known/webfinger API to lookup user hrefs + +- deleteuser gmg subcommand now fails gracefully +- Removed a false download link from setup.py + +0.7.0 +===== + +**Do this to upgrade** + +1. Update to the latest release. If checked out from git, run: + ``git fetch && git checkout -q v0.7.0 && git submodule init && git submodule update`` +2. Make sure to run + ``./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate`` + +(NOTE: earlier versions of the 0.7.0 release instructions left out the +``git submodule init`` step! If you did an upgrade earlier based on +these instructions and your theme looks weirdly aligned, try running +the following:) + + ``git submodule init && git submodule update`` + +That's it, probably! If you run into problems, don't hesitate to +`contact us <http://mediagoblin.org/pages/join.html>`_ +(IRC is often best). + +**New features:** + +- New mobile upload API making use of the + `Pump API <https://github.com/e14n/pump.io/blob/master/API.md>`_ + (which will be the foundation for MediaGoblin's federation) +- New theme: Sandy 70s Speedboat! + +- Metadata features! We also now have a JSON-LD context. + +- Many improvements for archival institutions, including metadata + support and featuring items on the homepage. With the (new!) + archivalook plugin enabled, featuring media is possible. + Additionally, metadata about the particular media item will show up + in the sidebar. + + In the future these plugins may be separated, but for now they have + come together as part of the same plugin. + +- There is a new gmg subcommand called batchaddmedia that allows for + uploading many files at once. This is aimed to be useful for + archival institutions and groups where there is an already existing + and large set of available media that needs to be included. +- Speaking of, the call to PostgreSQL in the makefile is fixed. +- We have a new, generic media-page context hook that allows for + adding context depending on the type of media. +- Tired of video thumbnails breaking during processing all the time? + Good news, everyone! Video thumbnail generation should not fail + frequently anymore. (We think...) +- You can now set default permissions for new users in the config. + +- bootstrap.sh / gnu configuration stuff still exists, but moves to be + experimental-bootstrap.sh so as to not confuse newcomers. There are + some problems currently with the autoconf stuff that we need to work + out... we still have interest in supporting it, though help is + welcome. + +- MediaGoblin now checks whether or not the database is up to date + when starting. +- Switched to `Skeleton <http://www.getskeleton.com/>`_ as a system for + graphic design. +- New gmg subcommands for administrators: + - A "deletemedia" command + - A "deleteuser" command +- We now have a blogging media type... it's very experimental, + however. Use with caution! +- We have switched to exifread as an external library for reading EXIF + data. It's basically the same thing as before, but packaged + separately from MediaGoblin. +- Many improvements to internationalization. Also (still rudimentary, + but existent!) RTL language support! + +**Known issues:** + - The host-meta is now JSON by default; in the spec it should be XML by + default. We have done this because of compatibility with the pump + API. We are checking with upstream to see if there is a way to + resolve this discrepancy. + + 0.6.1 ===== @@ -62,7 +360,7 @@ nickname "Lore of the Admin"! - New tools to control how much users can upload, both as a general user limit, or per file. - You can set this with the following options in your mediagoblin + You can set this with the following options in your MediaGoblin config file: `upload_limit` and `max_file_size`. Both are integers in megabytes. @@ -70,7 +368,7 @@ nickname "Lore of the Admin"! upload too, though an interface for this is not yet exposed. See the "uploaded" field on the core__users table. -- MediaGoblin now contains an authentication plugin for ldap! You +- MediaGoblin now contains an authentication plugin for LDAP! You can turn on the mediagoblin.plugins.ldap plugin to make use of this. See the documentation: :ref:`ldap-plugin` @@ -125,8 +423,8 @@ v0.5.1 is a bugfix release... the steps are the same as for 0.5.1. ===== **NOTE:** If using the API is important to you, we're in a state of -ransition towards a new API via the Pump API. As such, though the old -API still probably works, some changes have happened to the way oauth +transition towards a new API via the Pump API. As such, though the old +API still probably works, some changes have happened to the way OAuth works to make it more Pump-compatible. If you're heavily using clients using the old API, you may wish to hold off on upgrading for now. Otherwise, jump in and have fun! :) @@ -171,21 +469,21 @@ If you run into problems, don't hesitate to * Comment preview! * Users now have the ability to change their email associated with their account. -* New oauth code as we move closer to federation support. -* Experimental pyconfigure support for GNU-style configue and makefile +* New OAuth code as we move closer to federation support. +* Experimental pyconfigure support for GNU-style configure and makefile deployment. * Database foundations! You can now pre-populate the database models. * Way faster unit test run-time via in-memory database. * All mongokit stuff has been cleaned up. -* Fixes for non-ascii filenames. +* Fixes for non-ASCII filenames. * The option to stay logged in. -* Mediagoblin has been upgraded to use the latest `celery <http://celeryproject.org/>`_ +* MediaGoblin has been upgraded to use the latest `Celery <http://celeryproject.org/>`_ version. * You can now add jinja2 extensions to your config file to use in custom templates. * Fixed video permission issues. -* Mediagoblin docs are now hosted with multiple versions. -* We removed redundent tooltips from the STL media display. +* MediaGoblin docs are now hosted with multiple versions. +* We removed redundant tooltips from the STL media display. * We are now using itsdangerous for verification tokens. @@ -197,7 +495,7 @@ fix in the newly released document support which prevented the "conversion via libreoffice" feature. If you were running 0.4.0 you can upgrade to v0.4.1 via a simple -switch and restarting mediagoblin/celery with no other actions. +switch and restarting MediaGoblin/Celery with no other actions. Otherwise, follow 0.4.0 instructions. @@ -216,7 +514,7 @@ Otherwise, follow 0.4.0 instructions. Keep on reading to hear more about new plugin features. 4. If you want to take advantage of new plugins that have statically served assets, you are going to need to add the new "plugin_static" - section to your nginx config. Basically the following for nginx:: + section to your Nginx config. Basically the following for Nginx:: # Plugin static files (usually symlinked in) location /plugin_static/ { @@ -259,7 +557,7 @@ please note the following: date of an image when available (available as the "original_date_visible" variable) * Moved unit testing system from nosetests to py.test so we can better - handle issues with sqlalchemy exploding with different database + handle issues with SQLAlchemy exploding with different database configurations. Long story :) * You can now disable the ability to post comments. * Tags now can be up to length 255 characters by default. @@ -289,7 +587,7 @@ you run into problems, don't hesitate to * New dropdown menu for accessing various features. -* Significantly improved URL generation. Now mediagoblin won't give +* Significantly improved URL generation. Now MediaGoblin won't give up on making a slug if it looks like there will be a duplicate; it'll try extra hard to generate a meaningful one instead. @@ -297,13 +595,13 @@ you run into problems, don't hesitate to linking to a slug; /u/username/m/id:35/ is the kind of reference we now use to linking to entries with ids. However, old links with entries that linked to ids should work just fine with our migration. - The only urls that might break in this release are ones using colons + The only URLs that might break in this release are ones using colons or equal signs. * New template hooks for plugin authoring. * As a demonstration of new template hooks for plugin authoring, - openstreetmap support now moved to a plugin! + OpenStreetMap support now moved to a plugin! * Method to add media to collections switched from icon of paperclip to button with "add to collection" text. @@ -314,9 +612,9 @@ you run into problems, don't hesitate to waste gobs of memory. * Video transcoding now optional for videos that meet certain - criteria. By default, MediaGoblin will not transcode webm videos + criteria. By default, MediaGoblin will not transcode WebM videos that are smaller in resolution than the MediaGoblin defaults, and - MediaGoblin can also be configured to allow theora files to not be + MediaGoblin can also be configured to allow Theora files to not be transcoded as well. * Per-user license preference option; always want your uploads to be @@ -346,7 +644,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. **Do this to upgrade** - # directory of your mediagoblin install + # directory of your MediaGoblin install cd /srv/mediagoblin.example.org # copy source for this release @@ -366,13 +664,13 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. * **3d model support!** You can now upload STL and OBJ files and display them in - MediaGoblin. Requires a recent-ish Blender; for details see: + MediaGoblin. Requires a recent Blender; for details see: :ref:`deploying-chapter` * **trim_whitespace** We bundle the optional plugin trim_whitespace which reduces the size - of the delivered html output by reducing redundant whitespace. + of the delivered HTML output by reducing redundant whitespace. See :ref:`core-plugin-section` for plugin documentation @@ -386,7 +684,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. and `OMGMG <https://github.com/jwandborg/omgmg>`_, an example of a web application hooking up to the API. - This is a plugin, so you have to enable it in your mediagoblin + This is a plugin, so you have to enable it in your MediaGoblin config file by adding a section under [plugins] like:: [plugins] @@ -399,7 +697,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. For applications that use OAuth to connect to the API. - This is a plugin, so you have to enable it in your mediagoblin + This is a plugin, so you have to enable it in your MediaGoblin config file by adding a section under [plugins] like:: [plugins] @@ -419,7 +717,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. Geolocation is also now on by default. -* **Miscelaneous visual improvements** +* **Miscellaneous visual improvements** We've made a number of small visual improvements including newer and nicer looking thumbnails and improved checkbox placement. @@ -434,7 +732,7 @@ MongoDB-based MediaGoblin instance to the newer SQL-based system. 1. Make sure to run ``bin/gmg dbuptdate`` after upgrading. 2. If you set up your server config with an older version of - mediagoblin and the mediagoblin docs, it's possible you don't + MediaGoblin and the MediaGoblin docs, it's possible you don't have the "theme static files" alias, so double check to make sure that section is there if you are having problems. diff --git a/docs/source/siteadmin/theming.rst b/docs/source/siteadmin/theming.rst index 11ae3875..9c01a5b3 100644 --- a/docs/source/siteadmin/theming.rst +++ b/docs/source/siteadmin/theming.rst @@ -73,9 +73,9 @@ want to install this theme! Don't worry, it's fairly painless. Set up your webserver to serve theme assets ------------------------------------------- -If you followed the nginx setup example in :ref:`webserver-config` you +If you followed the Nginx setup example in :ref:`webserver-config` you should already have theme asset setup. However, if you set up your -server config with an older version of mediagoblin and the mediagoblin +server config with an older version of MediaGoblin and the MediaGoblin docs, it's possible you don't have the "theme static files" alias, so double check to make sure that section is there if you are having problems. @@ -103,7 +103,7 @@ Other variables you may consider setting: `theme_web_path` When theme-specific assets are specified, this is where MediaGoblin - will set the urls. By default this is ``"/theme_static/"`` so in + will set the URLs. By default this is ``"/theme_static/"`` so in the case that your theme was trying to access its file ``"images/shiny_button.png"`` MediaGoblin would link to ``/theme_static/images/shiny_button.png``. @@ -136,7 +136,7 @@ if necessary):: | | |- im_a_hedgehog.png # hedgehog-containing image used by theme | | '- custom_logo.png # your theme's custom logo | '- css/ - | '- hedgehog.css # your site's hedgehog-specific css + | '- hedgehog.css # your site's hedgehog-specific CSS |- README.txt # Optionally, a readme file (not required) |- AGPLv3.txt # AGPL license file for your theme. (good practice) '- CC0_1.0.txt # CC0 1.0 legalcode for the assets [if appropriate!] @@ -164,7 +164,7 @@ Only a few things need to go in here:: [theme] name = Hedgehog-ification description = For hedgehog lovers ONLY - licensing = AGPLv3 or later templates; assets (images/css) waived under CC0 1.0 + licensing = AGPLv3 or later templates; assets (images/CSS) waived under CC0 1.0 The name and description fields here are to give users an idea of what your theme is about. For the moment, we don't have any listing @@ -232,7 +232,7 @@ You should include AGPLv3.txt with your theme as this is required for the assets. You can copy this from ``mediagoblin/licenses/``. In the above example, we also use CC0 to waive our copyrights to -images and css, so we also included CC0_1.0.txt +images and CSS, so we also included CC0_1.0.txt A README.txt file @@ -247,7 +247,7 @@ Simple theming by adding CSS ---------------------------- Many themes won't require anything other than the ability to override -some of MediaGoblin's core css. Thankfully, doing so is easy if you +some of MediaGoblin's core CSS. Thankfully, doing so is easy if you combine the above steps! In your theme, do the following (make sure you make the necessary @@ -260,7 +260,7 @@ Great, now open that file and add something like this at the end:: <link rel="stylesheet" type="text/css" href="{{ request.staticdirect('/css/theme.css', 'theme') }}"/> -You can name the css file whatever you like. Now make the directory +You can name the CSS file whatever you like. Now make the directory for ``assets/css/`` and add the file ``assets/css/theme.css``. You can now put custom CSS files in here and any CSS you add will |