aboutsummaryrefslogtreecommitdiffstats
path: root/mediagoblin/storage
Commit message (Collapse)AuthorAgeFilesLines
* Apply pyupgrade --py36-plus.Ben Sturmfels2021-09-231-1/+1
| | | | This removes some 'u' prefixes and converts simple format() calls to f-strings.
* Remove remaining imports/calls to six not automatically removed by pyupgrade.Ben Sturmfels2021-03-053-5/+1
|
* Apply `pyupgrade --py3-plus` to remove Python 2 compatibility code.Ben Sturmfels2021-03-054-14/+11
|
* Fix more print statements. Refs #5331Berker Peksag2015-06-251-2/+4
|
* Fix bug 647Boris Bobrov2015-03-022-22/+11
| | | | | Reading a file into memory resulted in depletion of memory. Now files are read and written lazily, by chunks.
* Fix urlparse import.Berker Peksag2014-07-281-1/+1
|
* Cleanup mediagoblin._compat and fix translation tests.Berker Peksag2014-07-281-5/+5
|
* The file() builtin has been removed in Python 3. Use open() instead.Berker Peksag2014-06-072-4/+4
|
* Use six.text_type instead of unicode().Berker Peksag2014-06-021-1/+1
| | | | I will be switch to use ``from __future__ import unicode_literals`` later.
* Use six.iteritems() instead of dict.iteritems().Berker Peksag2014-05-262-2/+6
|
* Continue to port GMG codebase.Berker Peksag2014-05-082-2/+5
|
* added user upload limitsRodney Ewing2013-08-263-0/+16
|
* CloudFiles: Default to SSL URIsJoar Wandborg2013-06-071-1/+1
|
* Merge remote-tracking branch 'cwebber/254_delete_queue_directories'Elrond2013-04-082-5/+37
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | * cwebber/254_delete_queue_directories: Removing docstring bit about delete_file possibly deleting directories in the future Convert media processing backends to delete the queue directory (#254) Implement delete_dir in the FileStorage Conflicts: mediagoblin/media_types/image/processing.py mediagoblin/media_types/video/processing.py Conflicts, because those media_types already use the newer proc_state.delete_queue_file() method (which needs updating.
| * Removing docstring bit about delete_file possibly deleting directories in ↵Christopher Allan Webber2013-01-221-4/+0
| | | | | | | | | | | | | | | | the future I agree that delete_dir as a separate operation is a better way to do things, especially since there is a non-recursive deletion option that will politely fail if the directory is not empty.
| * Implement delete_dir in the FileStorageSebastian Spaeth2013-01-222-3/+39
| | | | | | | | | | | | | | | | plus options for deleting only empty directories and deleting them recursively. Not sure how cloudfile storage is or should be handled here. Are things such as a "directory" even a concept there?
* | Make copying to/from storage systems memory efficient (#419)Sebastian Spaeth2013-03-032-7/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | The copy_locally and copy_local_to_storage (very inconsistent terms BTW) were simply slurping in everything in RAM and writing it out at once. (the copy_locally was actually memory efficient if the remote system was local) Use shutil.copyfileobj which does chunked reads/writes on file objects. The default buffer size is 16kb, and as each chunk means a separate HTTP request for e.g. cloudfiles, we use a chunksize of 4MB here (which has just been arbitrarily set by me without tests). This should help with the failure to upload large files issue #419.
* | Added .send method to cloudfiles storage objectJoar Wandborg2013-03-031-1/+9
| | | | | | | | | | .. wrapper. Also added some logging - A .warn() for the legacy .write() method and a .debug() for the new copy_local_to_storage()
* | Make Cloudfiles copy memory efficient too (#419)Sebastian Spaeth2013-03-021-0/+48
|/ | | | | | | | | | | | | | | | It seems that (our implementation of) cloudfiles.write() takes all existing data and appends write(data) to it, sending the full monty over the wire everytime. This would of course absolutely kill chunked writes with some O(1^n) performance and bandwidth usage. So, override this method and use the Cloudfile's "send" interface instead. Also make the Cloudfile file wrapper an iterator that allows us to simply do "for data in cloudfile:" which will stream the data in a memory-efficient way. DO NOTE THAT THIS PATCH IS COMPLETELY UNTESTED DUE TO LACK OF SETUP PLEASE REVIEW AND VERIFY.
* Fixed attachmentsJoar Wandborg2012-10-231-1/+4
|
* Non-performance improvements to cloudfilesJoar Wandborg2012-08-051-8/+27
| | | | Improved documentation and added logging to cloudfiles
* Make filestorage available to code that only imports storage.Brett Smith2012-07-081-0/+2
| | | | The storage-related tests were relying on this, if nothing else.
* Fix problems from pyflakes outputWill Kahn-Greene2012-06-033-5/+6
|
* Also make sure mimetype here is set correctly via mimetype[0] (Thanks Shackra!)Christopher Allan Webber2012-04-181-1/+1
|
* Save the MIMEtype to cloudfiles correctly.Shackra2012-04-161-0/+6
|
* It's 2012 all up in hereChristopher Allan Webber2012-02-024-4/+4
|
* Merge branch 'video_gstreamer-only'Joar Wandborg2011-11-211-2/+8
|\ | | | | | | | | | | Conflicts: mediagoblin/db/migrations.py mediagoblin/submit/views.py
| * Multimedia support - Commiting from a not yet finished state - Details belowJoar Wandborg2011-09-231-2/+8
| | | | | | | | | | | | | | | | | | | | | | * DONE Initially testing with arista ** DONE Video display templates *** TODO Multi-browser support ** TODO Video thumbnails ** TODO Link to original video ** TODO Video cropping Also contains a lot of "debug" print's
* | Slightly clearer docs on copy_local_to_storageChristopher Allan Webber2011-11-201-0/+4
| |
* | copy_local_to_storage, both general and specialized-for-BasicFileStorage ↵Christopher Allan Webber2011-11-202-0/+22
| | | | | | | | | | | | | | versions This utility should allow for easy copying from a local filesystem to the storage instance.
* | Merge remote-tracking branch 'remotes/nyergler/pep8-ification'Christopher Allan Webber2011-11-131-0/+1
|\ \ | | | | | | | | | | | | | | | | | | | | | Conflicts: mediagoblin/db/migrations.py mediagoblin/db/models.py mediagoblin/user_pages/views.py mediagoblin/util.py
| * | Whitespace and formatting cleanup.Nathan Yergler2011-10-011-0/+1
| |/ | | | | | | | | | | | | * Removed trailing whitespace * Line length < 80 where possible * Honor conventions on number of blank lines * Honor conventions about spaces around :, =
* | Merge remote branch 'remotes/aaronw/bug444_fix_utils_py_redux'Christopher Allan Webber2011-10-011-2/+2
|\ \ | | | | | | | | | | | | Conflicts: mediagoblin/util.py
| * | Finished splitting util.py into separate files.Aaron Williamson2011-10-011-2/+2
| |/
* / mountstorage - Changed typo in importJoar Wandborg2011-09-151-1/+1
|/
* Feature #587 - Split storage.py into submodulesJoar Wandborg2011-09-124-0/+630
* Removed storage.py * Created submodules for filestorage, cloudfiles, mountstorage * Changed test_storage to reflect the changes made in the storage module structure * Added mediagoblin.storage.filestorage.BasicFileStorage as a default for both publicstore and queuestore's `storage_class`
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-03 16:26:56 +0000