Page MenuHomeClusterLabs Projects
Diviner Tech Docs PhabricatorChunkedFileStorageEngine

final class PhabricatorChunkedFileStorageEngine
Phorge Technical Documentation (Files)

This class is not documented.

Tasks

Constructing an Engine

  • final public function __construct() — Construct a new storage engine.

Engine Metadata

Managing File Data

Loading Storage Engines

Other Methods

Methods

final public function __construct()
Inherited

PhabricatorFileStorageEngine

Construct a new storage engine.

Return
this//Implicit.//

public function getEngineIdentifier()

PhabricatorFileStorageEngine

Return a unique, nonempty string which identifies this storage engine. This is used to look up the storage engine when files needs to be read or deleted. For instance, if you store files by giving them to a duck for safe keeping in his nest down by the pond, you might return 'duck' from this method.

PhabricatorChunkedFileStorageEngine
This method is not documented.
Return
stringUnique string for this engine, max length 32.

public function getEnginePriority()

PhabricatorFileStorageEngine

Prioritize this engine relative to other engines.

Engines with a smaller priority number get an opportunity to write files first. Generally, lower-latency filestores should have lower priority numbers, and higher-latency filestores should have higher priority numbers. Setting priority to approximately the number of milliseconds of read latency will generally produce reasonable results.

In conjunction with filesize limits, the goal is to store small files like profile images, thumbnails, and text snippets in lower-latency engines, and store large files in higher-capacity engines.

PhabricatorChunkedFileStorageEngine
This method is not documented.
Return
floatEngine priority.

public function canWriteFiles()

PhabricatorFileStorageEngine

Return true if the engine is currently writable.

Engines that are disabled or missing configuration should return false to prevent new writes. If writes were made with this engine in the past, the application may still try to perform reads.

PhabricatorChunkedFileStorageEngine

We can write chunks if we have at least one valid storage engine underneath us.

Return
boolTrue if this engine can support new writes.

public function hasFilesizeLimit()

PhabricatorFileStorageEngine

Return true if the engine has a filesize limit on storable files.

The getFilesizeLimit() method can retrieve the actual limit. This method just removes the ambiguity around the meaning of a 0 limit.

PhabricatorChunkedFileStorageEngine
This method is not documented.
Return
bool`true` if the engine has a filesize limit.

public function getFilesizeLimit()
Inherited

PhabricatorFileStorageEngine

Return maximum storable file size, in bytes.

Not all engines have a limit; use getFilesizeLimit() to check if an engine has a limit. Engines without a limit can store files of any size.

By default, engines define a limit which supports chunked storage of large files. In most cases, you should not change this limit, even if an engine has vast storage capacity: chunked storage makes large files more manageable and enables features like resumable uploads.

Return
intMaximum storable file size, in bytes.

public function isTestEngine()
Inherited

PhabricatorFileStorageEngine

Identifies storage engines that support unit tests.

These engines are not used for production writes.

Return
boolTrue if this is a test engine.

public function isChunkEngine()

PhabricatorFileStorageEngine

Identifies chunking storage engines.

If this is a storage engine which splits files into chunks and stores the chunks in other engines, it can return true to signal that other chunking engines should not try to store data here.

PhabricatorChunkedFileStorageEngine
This method is not documented.
Return
boolTrue if this is a chunk engine.

public function writeFile($data, $params)

PhabricatorFileStorageEngine

Write file data to the backing storage and return a handle which can later be used to read or delete it. For example, if the backing storage is local disk, the handle could be the path to the file.

The caller will provide a $params array, which may be empty or may have some metadata keys (like "name" and "author") in it. You should be prepared to handle writes which specify no metadata, but might want to optionally use some keys in this array for debugging or logging purposes. This is the same dictionary passed to PhabricatorFile::newFromFileData(), so you could conceivably do custom things with it.

If you are unable to write for whatever reason (e.g., the disk is full), throw an exception. If there are other satisfactory but less-preferred storage engines available, they will be tried.

PhabricatorChunkedFileStorageEngine
This method is not documented.
Parameters
string$dataThe file data to write.
array$paramsFile metadata (name, author), if available.
Return
stringUnique string which identifies the stored file, max length 255.

public function readFile($handle)

PhabricatorFileStorageEngine

Read the contents of a file previously written by writeFile().

PhabricatorChunkedFileStorageEngine
This method is not documented.
Parameters
string$handleThe handle returned from @{method:writeFile} when the file was written.
Return
stringFile contents.

public function deleteFile($handle)

PhabricatorFileStorageEngine

Delete the data for a file previously written by writeFile().

PhabricatorChunkedFileStorageEngine
This method is not documented.
Parameters
string$handleThe handle returned from @{method:writeFile} when the file was written.
Return
void

public static function loadStorageEngines($length)
Inherited

PhabricatorFileStorageEngine

Select viable default storage engines according to configuration. We'll select the MySQL and Local Disk storage engines if they are configured to allow a given file.

Parameters
int$lengthFile size in bytes.
Return
wild

public static function loadAllEngines()
Inherited

This method is not documented.
Return
wild

private static function loadProductionEngines()
Inherited

This method is not documented.
Return
wild

public static function loadWritableEngines()
Inherited

This method is not documented.
Return
wild

public static function loadWritableChunkEngines()
Inherited

This method is not documented.
Return
wild

public static function getChunkThreshold()
Inherited

PhabricatorFileStorageEngine

Return the largest file size which can not be uploaded in chunks.

Files smaller than this will always upload in one request, so clients can safely skip the allocation step.

Return
int|nullByte size, or `null` if there is no chunk support.

public function getRawFileDataIterator($file, $begin, $end, $format)

This method is not documented.
Parameters
PhabricatorFile$file
$begin
$end
PhabricatorFileStorageFormat$format
Return
wild

public function newIntegrityHash($data, $format)
Inherited

This method is not documented.
Parameters
$data
PhabricatorFileStorageFormat$format
Return
wild

private function loadAllChunks($handle, $need_files)

This method is not documented.
Parameters
$handle
$need_files
Return
wild

public static function getChunkedHash($viewer, $hash)

Compute a chunked file hash for the viewer.

We can not currently compute a real hash for chunked file uploads (because no process sees all of the file data).

We also can not trust the hash that the user claims to have computed. If we trust the user, they can upload some evil.exe and claim it has the same file hash as good.exe. When another user later uploads the real good.exe, we'll just create a reference to the existing evil.exe. Users who download good.exe will then receive evil.exe.

Instead, we rehash the user's claimed hash with account secrets. This allows users to resume file uploads, but not collide with other users.

Ideally, we'd like to be able to verify hashes, but this is complicated and time consuming and gives us a fairly small benefit.

Parameters
PhabricatorUser$viewerViewing user.
string$hashClaimed file hash.
Return
stringRehashed file hash.

public static function getChunkedHashForInput($input)

This method is not documented.
Parameters
$input
Return
wild

public function allocateChunks($length, $properties)

This method is not documented.
Parameters
$length
array$properties
Return
wild

private function getWritableEngine()

Find a storage engine which is suitable for storing chunks.

This engine must be a writable engine, have a filesize limit larger than the chunk limit, and must not be a chunk engine itself.

Return
wild

public function getChunkSize()

This method is not documented.
Return
wild