Page MenuHomeClusterLabs Projects
Diviner Tech Docs PhabricatorS3FileStorageEngine

final class PhabricatorS3FileStorageEngine
Phorge Technical Documentation (Files)

Amazon S3 file storage engine. This engine scales well but is relatively high-latency since data has to be pulled off S3.

Tasks

Constructing an Engine

  • final public function __construct() — Construct a new storage engine.

Engine Metadata

Managing File Data

  • public function writeFile($data, $params) — Writes file data into Amazon S3.
  • public function readFile($handle) — Load a stored blob from Amazon S3.
  • public function deleteFile($handle) — Delete a blob from Amazon S3.

Loading Storage Engines

Internals

  • private function getBucketName() — Retrieve the S3 bucket name.
  • private function newS3API() — Create a new S3 API object.

Other Methods

Methods

final public function __construct()
Inherited

PhabricatorFileStorageEngine

Construct a new storage engine.

Return
this//Implicit.//

public function getEngineIdentifier()

PhabricatorFileStorageEngine

Return a unique, nonempty string which identifies this storage engine. This is used to look up the storage engine when files needs to be read or deleted. For instance, if you store files by giving them to a duck for safe keeping in his nest down by the pond, you might return 'duck' from this method.

PhabricatorS3FileStorageEngine

This engine identifies as amazon-s3.

Return
stringUnique string for this engine, max length 32.

public function getEnginePriority()

PhabricatorFileStorageEngine

Prioritize this engine relative to other engines.

Engines with a smaller priority number get an opportunity to write files first. Generally, lower-latency filestores should have lower priority numbers, and higher-latency filestores should have higher priority numbers. Setting priority to approximately the number of milliseconds of read latency will generally produce reasonable results.

In conjunction with filesize limits, the goal is to store small files like profile images, thumbnails, and text snippets in lower-latency engines, and store large files in higher-capacity engines.

PhabricatorS3FileStorageEngine
This method is not documented.
Return
floatEngine priority.

public function canWriteFiles()

PhabricatorFileStorageEngine

Return true if the engine is currently writable.

Engines that are disabled or missing configuration should return false to prevent new writes. If writes were made with this engine in the past, the application may still try to perform reads.

PhabricatorS3FileStorageEngine
This method is not documented.
Return
boolTrue if this engine can support new writes.

public function hasFilesizeLimit()
Inherited

PhabricatorFileStorageEngine

Return true if the engine has a filesize limit on storable files.

The getFilesizeLimit() method can retrieve the actual limit. This method just removes the ambiguity around the meaning of a 0 limit.

Return
bool`true` if the engine has a filesize limit.

public function getFilesizeLimit()
Inherited

PhabricatorFileStorageEngine

Return maximum storable file size, in bytes.

Not all engines have a limit; use getFilesizeLimit() to check if an engine has a limit. Engines without a limit can store files of any size.

By default, engines define a limit which supports chunked storage of large files. In most cases, you should not change this limit, even if an engine has vast storage capacity: chunked storage makes large files more manageable and enables features like resumable uploads.

Return
intMaximum storable file size, in bytes.

public function isTestEngine()
Inherited

PhabricatorFileStorageEngine

Identifies storage engines that support unit tests.

These engines are not used for production writes.

Return
boolTrue if this is a test engine.

public function isChunkEngine()
Inherited

PhabricatorFileStorageEngine

Identifies chunking storage engines.

If this is a storage engine which splits files into chunks and stores the chunks in other engines, it can return true to signal that other chunking engines should not try to store data here.

Return
boolTrue if this is a chunk engine.

public function writeFile($data, $params)

PhabricatorFileStorageEngine

Write file data to the backing storage and return a handle which can later be used to read or delete it. For example, if the backing storage is local disk, the handle could be the path to the file.

The caller will provide a $params array, which may be empty or may have some metadata keys (like "name" and "author") in it. You should be prepared to handle writes which specify no metadata, but might want to optionally use some keys in this array for debugging or logging purposes. This is the same dictionary passed to PhabricatorFile::newFromFileData(), so you could conceivably do custom things with it.

If you are unable to write for whatever reason (e.g., the disk is full), throw an exception. If there are other satisfactory but less-preferred storage engines available, they will be tried.

PhabricatorS3FileStorageEngine

Writes file data into Amazon S3.

Parameters
string$dataThe file data to write.
array$paramsFile metadata (name, author), if available.
Return
stringUnique string which identifies the stored file, max length 255.

public function readFile($handle)

PhabricatorFileStorageEngine

Read the contents of a file previously written by writeFile().

PhabricatorS3FileStorageEngine

Load a stored blob from Amazon S3.

Parameters
string$handleThe handle returned from @{method:writeFile} when the file was written.
Return
stringFile contents.

public function deleteFile($handle)

PhabricatorFileStorageEngine

Delete the data for a file previously written by writeFile().

PhabricatorS3FileStorageEngine

Delete a blob from Amazon S3.

Parameters
string$handleThe handle returned from @{method:writeFile} when the file was written.
Return
void

public static function loadStorageEngines($length)
Inherited

PhabricatorFileStorageEngine

Select viable default storage engines according to configuration. We'll select the MySQL and Local Disk storage engines if they are configured to allow a given file.

Parameters
int$lengthFile size in bytes.
Return
wild

public static function loadAllEngines()
Inherited

This method is not documented.
Return
wild

private static function loadProductionEngines()
Inherited

This method is not documented.
Return
wild

public static function loadWritableEngines()
Inherited

This method is not documented.
Return
wild

public static function loadWritableChunkEngines()
Inherited

This method is not documented.
Return
wild

public static function getChunkThreshold()
Inherited

PhabricatorFileStorageEngine

Return the largest file size which can not be uploaded in chunks.

Files smaller than this will always upload in one request, so clients can safely skip the allocation step.

Return
int|nullByte size, or `null` if there is no chunk support.

public function getRawFileDataIterator($file, $begin, $end, $format)
Inherited

This method is not documented.
Parameters
PhabricatorFile$file
$begin
$end
PhabricatorFileStorageFormat$format
Return
wild

public function newIntegrityHash($data, $format)
Inherited

This method is not documented.
Parameters
$data
PhabricatorFileStorageFormat$format
Return
wild

private function getBucketName()

Retrieve the S3 bucket name.

Return
wild

private function newS3API()

Create a new S3 API object.

Return
wild