Sign up Sign in Samples Blog contact support
file storage options

Ticket

 
+issue

our current upload functionality is optimized for security, but because it intertwines the app server and file server roles, it makes multi server architecture more challenging.  It also uses resources for permissions checking even on public files/images.  Moreover, it makes it impossible to use service like heroku.

 

So...we need to support cloud files, and ideally we should be able to support both the current and other versions.

 

+solution

Wagn should support these mechanisms for viewing files/images.   Note that even different revisions of the same card can use different mechanisms.

  1. coded files - db_content starts with ':'.  These files are in the codebase, like the default logo.  Every view is a wagn request. 
  2. local files – db_content starts with '~'.  These are in the deck's "files" directory by default.  If read permissions are set such that "Anyone" can read, then there is a symlink from the public directory.  Otherwise every view is a wagn request. 
  3. "bucketed" files - db_content starts with '('.  Eg "(mybucket)/123/4567.png".   bucket is substituted for url base.  rendered as absolute url
  4. (?) complete urls - db_content starts with http (or other?).    rendered as is.

In 3-5, the content may be stored on the same server or another.   In general  none of the three should lead to wagn requests.

 

They correspond to these mechanisms for storing cards:

  1. coded files are handled by coders (eg with rake tasks)
  2. protected uploaded files are the default.  (current functionality)
  3. linking/unlinking to unprotected files is handled on permission-changing events. 
  4. this is standard cloud upload behavior

Unlike viewing files/images, which can all be options in the same deck, any given deck will need to have a single default storage option for creating/updating cards.  (One configuration option in the future might present a dropdown for the user, but for now there is not much demand for that; it's much more important to keep things clean and simple).

Configuration

the explicit (but unnecessary) configuration of current behavior

config.file_storage :protected  # default

 

buckets are a bit more involved: 

config.file_storage :bucket1
config.file_buckets({
bucket1: {
provider: ...
credentials: ...
directory: ...
},
bucket2: {
url_base: ...
}
})

 bucket1 above is an example of a full-fledged cloud service.  Ready for createing and updating files/images.

bucket2 is basically read-only.  It would not be able to create/update images, but it would let folks render files and images and may help us support some cool data sharing patterns.  (Eg, you could publish a set of image variants and switch to different buckets to get the different variants.  Or you could have a dev server that didn't let you accidentally alter images on the cloud but let you see them.  Or you could let someone use your "hand" of cards but not given them access to edit its images/files.

 

 

 

 

 

+example

one valuable use case: development copies of a public website (eg WikiRate)

  • live site needs write access to live bucket.  we could, in theory, turn off delete access to reduce risk of accidental data loss.
  • dev site needs configuration for read access to live bucket so that the site can use images from the live site
  • typically speaking, we probably want to have a separate dev bucket, so that when we test things on dev, we're not adding nonsense to the live bucket.
  • same general pattern applies to demo, staging...
  • however, we may eventually also want a solution for dev that lets us commit files/images to the live site if/when we know we want them to be available.  (not needed right away)

another valuable use case: Heroku deployments should work with this configuration!

 

 

 

+Relevant user stories