Diffusion User Guide: Symbol Indexes

Phorge Administrator and User Documentation (Application User Guides)

Guide to configuring and using the symbol index.

Overview

Phorge can maintain a symbol index, which keeps track of where classes and functions are defined in the codebase. Once you set up indexing, you can use the index to do things like:

• jump to symbol definitions from Differential code reviews and Diffusion code browsing by ctrl-clicking (cmd-click on Mac) symbols
• search for symbols from the quick-search
• let the IRC bot answer questions like "Where is SomeClass?"

Populating the Index

To populate the index, you need to write a script which identifies symbols in your codebase and set up a cronjob which pipes its output to:

./scripts/symbols/import_repository_symbols.php

Phorge includes a script which can identify symbols in PHP projects:

./scripts/symbols/generate_php_symbols.php

Phorge also includes a script which can identify symbols in any programming language that has classes and/or functions, and is supported by Exuberant Ctags (http://ctags.sourceforge.net):

./scripts/symbols/generate_ctags_symbols.php

If you want to identify symbols from another language, you need to write a script which can export them (for example, maybe by parsing a ctags file).

The output format of the script should be one symbol per line:

<context> <name> <type> <lang> <line> <path>

For example:

ExampleClass exampleMethod function php 13 /src/classes/ExampleClass.php

Context is, broadly speaking, the scope or namespace where the symbol is defined. For object-oriented languages, this is probably a class name. The symbols with that context are class constants, methods, properties, nested classes, etc. When printing symbols without a context (those that are defined globally, for instance), the <context> field should be empty (that is, the line should start with a space).

Your script should enumerate all the symbols in your project, and provide paths from the project root (where ".arcconfig" is) beginning with a "/".

You can look at generate_php_symbols.php for an example of how you might write such a script, and run this command to see its output:

$ cd phorge/
$ find . -type f -name '*.php' | ./scripts/symbols/generate_php_symbols.php

To actually build the symbol index, pipe this data to the import_repository_symbols.php script, providing the repository callsign:

$ ./scripts/symbols/import_repository_symbols.php REPO < symbols_data

Then just set up a cronjob to run that however often you like.

You can test that the import worked by querying for symbols using the Conduit method diffusion.findsymbols. Some features (like that method, and the IRC bot integration) will start working immediately. Others will require more configuration.

Advanced Configuration

You can configure some more options by going to Diffusion → (Select repository) → Edit Repository → Edit Symbols, and filling out these fields:

• Indexed Languages: Fill in all the languages you've built indexes for. You can leave this blank for "All languages".
• Uses Symbols From: If this project depends on other repositories, add the other repositories which symbols should be looked for here. For example, Phorge lists "Arcanist" because it uses classes and functions defined in arcanist/.

External Symbols

By Adding New Classes, you can teach Phorge about symbols from the outside world. Extend DiffusionExternalSymbolsSource; Once loaded, your new implementation will be used any time a symbol is queried.

See DiffusionPhpExternalSymbolsSource and DiffusionPythonExternalSymbolsSource for example implementations.