diff options
Diffstat (limited to 'docs/source/devel')
| -rw-r--r-- | docs/source/devel/codebase.rst | 22 | ||||
| -rw-r--r-- | docs/source/devel/migrations.rst | 46 | ||||
| -rw-r--r-- | docs/source/devel/originaldesigndecisions.rst | 10 | ||||
| -rw-r--r-- | docs/source/devel/storage.rst | 24 |
4 files changed, 49 insertions, 53 deletions
diff --git a/docs/source/devel/codebase.rst b/docs/source/devel/codebase.rst index 122a3297..068c1873 100644 --- a/docs/source/devel/codebase.rst +++ b/docs/source/devel/codebase.rst @@ -44,7 +44,7 @@ development. What's where ============ -After you've run checked out mediagoblin and followed the virtualenv +After you've run checked out MediaGoblin and followed the virtualenv instantiation instructions, you're faced with the following directory tree:: @@ -86,8 +86,8 @@ As you can see, all the code for GNU MediaGoblin is in the Here are some interesting files and what they do: -:routing.py: maps url paths to views -:views.py: views handle http requests +:routing.py: maps URL paths to views +:views.py: views handle HTTP requests :forms.py: wtforms stuff for this submodule You'll notice that there are several sub-directories: tests, @@ -97,7 +97,7 @@ templates, auth, submit, ... ``templates`` holds all the templates for the output. -``auth`` and ``submit`` are modules that enacpsulate authentication +``auth`` and ``submit`` are modules that encapsulate authentication and media item submission. If you look in these directories, you'll see they have their own ``routing.py``, ``view.py``, and forms.py in addition to some other code. @@ -123,15 +123,15 @@ Software Stack for unit tests * `virtualenv <http://www.virtualenv.org/>`_: for setting up an - isolated environment to keep mediagoblin and related packages + isolated environment to keep MediaGoblin and related packages (potentially not required if MediaGoblin is packaged for your distro) * Data storage * `SQLAlchemy <http://sqlalchemy.org/>`_: SQL ORM and database - interaction library for Python. Currently we support sqlite and - postgress as backends. + interaction library for Python. Currently we support SQLite and + PostgreSQL as backends. * Web application @@ -158,10 +158,10 @@ Software Stack * `Markdown (for python) <http://pypi.python.org/pypi/Markdown>`_: implementation of `Markdown <http://daringfireball.net/projects/markdown/>`_ - text-to-html tool to make it easy for people to write richtext + text-to-html tool to make it easy for people to write rich text comments, descriptions, and etc. - * `lxml <http://lxml.de/>`_: nice xml and html processing for + * `lxml <http://lxml.de/>`_: nice XML and HTML processing for python. * Media processing libraries @@ -174,10 +174,10 @@ Software Stack future, probably audio too. * `chardet <http://pypi.python.org/pypi/chardet>`_: (Optional, for - ascii art hosting sites only) Used to make ascii art thumbnails. + ASCII art hosting sites only) Used to make ASCII art thumbnails. * Front end - * `JQuery <http://jquery.com/>`_: for groovy JavaScript things + * `jQuery <http://jquery.com/>`_: for groovy JavaScript things diff --git a/docs/source/devel/migrations.rst b/docs/source/devel/migrations.rst index 16c02b04..ec0f7d29 100644 --- a/docs/source/devel/migrations.rst +++ b/docs/source/devel/migrations.rst @@ -28,35 +28,35 @@ every migration is run with dbupdate. There's a few things you need to know: -- We use `sqlalchemy-migrate +- We use `Alembic <https://bitbucket.org/zzzeek/alembic>`_ to run + migrations. We also make heavy use of the + `branching model <http://alembic.readthedocs.org/en/latest/branches.html>`_ + for our plugins. Every plugin gets its own migration branch. +- We used to use `sqlalchemy-migrate <http://code.google.com/p/sqlalchemy-migrate/>`_. See `their docs <https://sqlalchemy-migrate.readthedocs.org/>`_. -- `Alembic <https://bitbucket.org/zzzeek/alembic>`_ might be a better - choice than sqlalchemy-migrate now or in the future, but we - originally decided not to use it because it didn't have sqlite - support. It's not clear if that's changed. + sqlalchemy-migrate is now only kept around for legacy migrations; + don't add any more! But some users are still using older databases, + and we need to provide the intermediary "old" migrations for a while. - SQLAlchemy has two parts to it, the ORM and the "core" interface. We DO NOT use the ORM when running migrations. Think about it: the ORM is set up with an expectation that the models already reflect a - certain pattern. But if a person is moving from their old patern + certain pattern. But if a person is moving from their old pattern and are running tools to *get to* the current pattern, of course their current database structure doesn't match the state of the ORM! -- How to write migrations? Maybe there will be a tutorial here in the - future... in the meanwhile, look at existing migrations in - `mediagoblin/db/migrations.py` and look in - `mediagoblin/tests/test_sql_migrations.py` for examples. -- Common pattern: use `inspect_table` to get the current state - of the table before we run alterations on it. -- Make sure you set the RegisterMigration to be the next migration in - order. -- What happens if you're adding a *totally new* table? In this case, - you should copy the table in entirety as it exists into - migrations.py then create the tables based off of that... see - add_collection_tables. This is easier than reproducing the SQL by - hand. -- If you're writing a feature branch, you don't need to keep adding - migrations every time you change things around if your database - structure is in flux. Just alter your migrations so that they're - correct for the merge into master. + Anyway, Alembic has its own conventions for migrations; follow those. +- Alembic's documentation is pretty good; you don't need to worry about + setting up the migration environment or the config file so you can + skip those parts. You can start at the + `Create a Migration Script <http://alembic.readthedocs.org/en/latest/tutorial.html#create-a-migration-script>`_ + section. +- *Users* should only use `./bin/gmg dbupdate`. However, *developers* + may wish to use the `./bin/gmg alembic` subcommand, which wraps + alembic's own command line interface. Alembic has some tools for + `autogenerating migrations <http://alembic.readthedocs.org/en/latest/autogenerate.html>`_, + and they aren't perfect, but they are helpful. (You can pass in + `./bin/gmg alembic --with-plugins revision --autogenerate` if you need + to include plugins in the generated output; see the + :ref:`plugin database chapter <plugin-database-chapter>` for more info.) That's it for now! Good luck! diff --git a/docs/source/devel/originaldesigndecisions.rst b/docs/source/devel/originaldesigndecisions.rst index 2843870c..b60db776 100644 --- a/docs/source/devel/originaldesigndecisions.rst +++ b/docs/source/devel/originaldesigndecisions.rst @@ -38,7 +38,7 @@ Chris and Will on "Why GNU MediaGoblin": .. figure:: ../_static/goblin.png :alt: Cute goblin with a beret. - *Figure 1: Cute goblin with a beret. llustrated by Chris + *Figure 1: Cute goblin with a beret. Illustrated by Chris Webber* .. figure:: ../_static/snugglygoblin.png @@ -61,7 +61,7 @@ Chris and Will on "Why GNU MediaGoblin": 1. "GNU MediaGoblin" is the name we're going to use in all official capacities: web site, documentation, press releases, ... - 2. In casual conversation, it's ok to use more casual names. + 2. In casual conversation, it's OK to use more casual names. 3. If you're writing about the project, we ask that you call it GNU MediaGoblin. @@ -113,7 +113,7 @@ Why WSGI Minimalism Chris Webber on "Why WSGI Minimalism": If you notice in the technology list I list a lot of components - that are very "django-like", but not actually `Django`_ + that are very "Django-like", but not actually `Django`_ components. What can I say, I really like a lot of the ideas in Django! Which leads to the question: why not just use Django? @@ -176,7 +176,7 @@ Chris Webber on "Why MongoDB": ideal universe where everyone ran servers out of their own housing. As a memory-mapped database, MongoDB is pretty hungry, so actually I spent a lot of time debating whether the inability - to scale down as nicely as something like SQL has with sqlite + to scale down as nicely as something like SQL has with SQLite meant that it was out. But I decided in the end that I really want MongoDB, not for @@ -199,7 +199,7 @@ Chris Webber on "Why MongoDB": Being able to just dump media-specific information in a media_data - hashtable is pretty great, and even better is having a plugin + hash table is pretty great, and even better is having a plugin system where you can just let plugins have their own entire key-value space cleanly inside the document that doesn't interfere with anyone else's stuff. If we were to let plugins to deposit diff --git a/docs/source/devel/storage.rst b/docs/source/devel/storage.rst index 215f9579..46e163de 100644 --- a/docs/source/devel/storage.rst +++ b/docs/source/devel/storage.rst @@ -21,7 +21,7 @@ are: + **public_store:** After your media goes through processing it gets moved to the public store. This is also a StorageInterface - implelementation, and is for stuff that's intended to be seen by + implementation, and is for stuff that's intended to be seen by site visitors. The workbench @@ -43,11 +43,9 @@ Static assets / staticdirect ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ On top of all that, there is some static media that comes bundled with your -application. This stuff is kept in: +application. This stuff is kept in ``mediagoblin/static/``. - mediagoblin/static/ - -These files are for mediagoblin base assets. Things like the CSS files, +These files are for MediaGoblin base assets. Things like the CSS files, logos, etc. You can mount these at whatever location is appropriate to you (see the direct_remote_path option in the config file) so if your users are keeping their static assets at http://static.mgoblin.example.org/ but @@ -55,7 +53,7 @@ their actual site is at http://mgoblin.example.org/, you need to be able to get your static files in a where-it's-mounted agnostic way. There's a "staticdirector" attached to the request object. It's pretty easy to use; just look at this bit taken from the -mediagoblin/templates/mediagoblin/base.html main template: +mediagoblin/templates/mediagoblin/base.html main template:: <link rel="stylesheet" type="text/css" href="Template:Request.staticdirect('/css/extlib/text.css')"/> @@ -76,9 +74,7 @@ So, the StorageInterface! So, the public and queue stores both use StorageInterface implementations ... but what does that mean? It's not too hard. -Open up: - - mediagoblin/storage.py +Open up ``mediagoblin/storage.py``. In here you'll see a couple of things. First of all, there's the StorageInterface class. What you'll see is that this is just a very simple @@ -95,12 +91,12 @@ also the default storage system used. As expected, this stores files locally on your machine. There's also a CloudFileStorage system. This provides a mapping to -[OpenStack's swift http://swift.openstack.org/] storage system (used by +[OpenStack's Swift http://swift.openstack.org/] storage system (used by RackSpace Cloud files and etc). Between these two examples you should be able to get a pretty good idea of how to write your own storage systems, for storing data across your -beowulf cluster of radioactive monkey brains, whatever. +Beowulf cluster of radioactive monkey brains, whatever. Writing code to store stuff ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -112,10 +108,10 @@ https://docs.djangoproject.com/en/dev/ref/files/storage/]... with some differences. Basically, you access files on "file paths", which aren't exactly like -unix file paths, but are close. If you wanted to store a file on a path -like dir1/dir2/filename.jpg you'd actually write that file path like: +Unix file paths, but are close. If you wanted to store a file on a path +like dir1/dir2/filename.jpg you'd actually write that file path like:: -['dir1', 'dir2', 'filename.jpg'] + ['dir1', 'dir2', 'filename.jpg'] This way we can be *sure* that each component is actually a component of the path that's expected... we do some filename cleaning on each component. |