Diffusion User Guide: URIs

Phorge Administrator and User Documentation (Application User Guides)

Guide to configuring repository URIs for fetching, cloning and mirroring.

Overview

Phorge can create, host, observe, mirror, proxy, and import repositories. For example, you can:

Host Repositories: Phorge can host repositories locally. Phorge maintains the writable master version of the repository, and you can push and pull the repository. This is the most straightforward kind of repository configuration, and similar to repositories on other services like GitHub or Bitbucket.

Observe Repositories: Phorge can create a copy of an repository which is hosted elsewhere (like GitHub or Bitbucket) and track updates to the remote repository. This will create a read-only copy of the repository in Phorge.

Mirror Repositories: Phorge can publish any repository to mirrors, overwriting them with an exact copy of the repository that stays up to date as the source changes. This works with both local repositories that Phorge is hosting and remote repositories that Phorge is observing.

Proxy Repositories: If you are observing a repository, you can allow users to read Phorge's copy of the repository. Phorge supports granular read permissions, so this can let you open a private repository up a little bit in a flexible way.

Import Repositories: If you have a repository elsewhere that you want to host on Phorge, you can observe the remote repository first, then turn the tracking off once the repository fully synchronizes. This allows you to copy an existing repository and begin hosting it in Phorge.

You can also import repositories by creating an empty hosted repository and then pushing everything to the repository directly.

You configure the behavior of a Phorge repository by adding and configuring URIs and marking them to be fetched from, mirrored to, clonable, and so on. By configuring all the URIs that a repository should interact with and expose to users, you configure the read, write, and mirroring behavior of the repository.

The remainder of this document walks through this configuration in greater detail.

Host a Repository

You can create new repositories that Phorge will host, like you would create repositories on services like GitHub or Bitbucket. Phorge will serve a read-write copy of the repository and you can clone it from Phorge and push changes to Phorge.

If you haven't already, you may need to configure Phorge for hosting before you can create your first hosted repository. For a detailed guide, see Diffusion User Guide: Repository Hosting.

This is the default mode for new repositories. To host a repository:

• Create a new repository.
• Activate it.

Phorge will create an empty repository and allow you to fetch from it and push to it.

Observe a Repository

If you have an existing repository hosted on another service (like GitHub, Bitbucket, or a private server) that you want to work with in Phorge, you can configure Phorge to observe it.

When observing a repository, Phorge will keep track of changes in the remote repository and allow you to browse and interact with the repository from the web UI in Diffusion and other applications, but you can continue hosting it elsewhere.

To observe a repository:

• Create a new repository, but don't activate it yet.
• Add the remote URI you want to observe as a repository URI.
• Set the I/O Type for the URI to Observe.
• If necessary, configure a credential.
• Activate the repository.

Phorge will perform an initial import of the repository, creating a local read-only copy. Once this process completes, it will continue keeping track of changes in the remote, fetching them, and reflecting them in the UI.

Mirror a Repository

You can create a read-only mirror of an existing repository. Phorge will continuously publish the state of the source repository to the mirror, creating an exact copy.

For example, if you have a repository hosted in Phorge that you want to mirror to GitHub, you can configure Phorge to automatically maintain the mirror. This is how the upstream repositories are set up.

The mirror copy must be read-only for users because any writes made to the mirror will be undone when Phorge updates it. The mirroring process copies the entire repository state exactly, so the remote state will be completely replaced with an exact copy of the source repository. This may remove or destroy information. Normally, you should only mirror to an empty repository.

You can mirror any repository, even if Phorge is only observing it and not hosting it directly.

To begin mirroring a repository:

• Create a hosted or observed repository by following the relevant instructions above.
• Add the remote URI you want to mirror to as a repository URI.
• Set the I/O Type for the URI to Mirror.
• If necessary, configure a credential.

To stop mirroring:

• Disable the mirror URI; or
• Change the I/O Type for the URI to None.

Import a Repository

If you have an existing repository that you want to move so it is hosted on Phorge, there are three ways to do it:

Observe First: (Git, Mercurial) Observe the existing repository first, according to the instructions above. Once Phorge's copy of the repository is fully synchronized, change the I/O Type for the Observe URI to None to stop fetching changes from the remote.

By default, this will automatically make Phorge's copy of the repository writable, and you can begin pushing to it. If you've adjusted URI configuration away from the defaults, you may need to set at least one URI to Read/Write mode so you can push to it.

Push Everything: (Git, Mercurial, Subversion) Create a new empty hosted repository according to the instructions above. Once the empty repository initializes, push your entire existing repository to it.

In Subversion, you can do this with the svnsync tool.

Copy on Disk: (Git, Mercurial, Subversion) Create a new empty hosted repository according to the instructions above, but do not activate it yet.

Using the Storage tab, find the location of the repository's working copy on disk, and place a working copy of the repository you wish to import there.

For Git and Mercurial, use a bare working copy for best results.

This is the only way to import a Subversion repository because only the master copy of the repository has history.

Once you've put a working copy in the right place on disk, activate the repository.

Builtin Clone URIs

By default, Phorge automatically exposes and activates HTTP, HTTPS and SSH clone URIs by examining configuration.

HTTP: The http:// clone URI will be available if these conditions are satisfied:

• diffusion.allow-http-auth must be enabled or the repository view policy must be "Public".
• The repository must be a Git or Mercurial repository.
• security.require-https must be disabled.

HTTPS: The https:// clone URI will be available if these conditions are satisfied:

• diffusion.allow-http-auth must be enabled or the repository view policy must be "Public".
• The repository must be a Git or Mercurial repository.
• The phabricator.base-uri protocol must be https://.

SSH: The ssh:// or svn+ssh:// clone URI will be available if these conditions are satisfied:

• phd.user must be configured.

Customizing Displayed Clone URIs

If you have an unusual configuration and want the UI to offers users specific clone URIs other than the URIs that Phorge serves or interacts with, you can add those URIs with the I/O Type set to None and then set their Display Type to Always.

Likewise, you can set the Display Type of any URIs you do not want to be visible to Never.

This allows you to precisely configure which clone URIs are shown to users for a repository.

Reference: I/O Types

This section details the available I/O Type options for URIs.

Each repository has some builtin URIs. These are URIs hosted by Phorge itself. The modes available for each URI depend primarily on whether it is a builtin URI or not.

Default: This setting has Phorge guess the correct option for the URI.

For builtin URIs, the default behavior is Read/Write if the repository is hosted, and Read-Only if the repository is observed.

For custom URIs, the default type is None because we can not automatically guess if you want to ignore, observe, or mirror a URI and None is the safest default.

Observe: Phorge will observe this repository and regularly fetch any changes made to it to a local read-only copy.

You can not observe builtin URIs because reading a repository from itself does not make sense.

You can not add a URI in Observe mode if an existing builtin URI is in Read/Write mode, because this would mean the repository had two different authorities: the observed remote copy and the hosted local copy. Take the other URI out of Read/Write mode first.

Mirror: Phorge will push any changes made to this repository to the remote URI, keeping a read-only mirror hosted at that URI up to date.

This works for both observed and hosted repositories.

This option is not available for builtin URIs because it does not make sense to mirror a repository to itself.

It is possible to mirror a repository to another repository that is also hosted by Phorge by adding that other repository's URI, although this is silly and probably very rarely of any use.

None: Phorge will not fetch changes from or push changes to this URI. For builtin URIs, it will not let users fetch changes from or push changes to this URI.

You can use this mode to turn off an Observe URI after an import, stop a Mirror URI from updating, or to add URIs that you're only using to customize which clone URIs are displayed to the user but don't want Phorge to interact with directly.

Read Only: Phorge will serve the repository from this URI in read-only mode. Users will be able to fetch from it but will not be able to push to it.

Because Phorge must be able to serve the repository from URIs configured in this mode, this option is only available for builtin URIs.

Read/Write: Phorge will serve the repository from this URI in read/write mode. Users will be able to fetch from it and push to it.

URIs can not be set into this mode if another URI is set to Observe mode, because that would mean the repository had two different authorities: the observed remote copy and the hosted local copy. Take the other URI out of Observe mode first.

Because Phorge must be able to serve the repository from URIs configured in this mode, this option is only available for builtin URIs.

Reference: Display Types

This section details the available Display Type options for URIs.

Default: Phorge will guess the correct option for the URI. It guesses based on the configured I/O Type and whether the URI is builtin or not.

For Observe, Mirror and None URIs, the default is Never.

For builtin URIs in Read Only or Read/Write mode, the most human-readable URI defaults to Always and the others default to Never.

Always: This URI will be shown to users as a clone/checkout URI. You can add URIs in this mode to customize exactly what users are shown.

Never: This URI will not be shown to users. You can hide less-preferred URIs to guide users to the URIs they should be using to interact with the repository.

Next Steps

Continue by:

• configuring Phorge to host repositories with Diffusion User Guide: Repository Hosting.