Writing plug-ins
Writing plug-ins for mirror.py
mirror.py uses a PEP 621 entry-points based plug-in system. Plug-ins are regular pip-installable Python packages — discovery happens through importlib.metadata.entry_points(), not by listing file paths in config.
If you have written pytest, Sphinx, or Celery plug-ins before, the model will feel familiar.
Three plug-in categories
Category |
Entry-point group |
What it does |
Example use case |
|---|---|---|---|
|
|
Implements a new synctype (in addition to built-in |
Mirror via SFTP, S3, custom HTTP transport |
|
|
Subscribes to mirror events to perform side effects |
Slack/email notification on sync failure, push metrics to Prometheus, custom audit log |
|
|
Contributes extra fields into |
Bandwidth-used counter, freshness classification, mirror health rollup |
A single distribution (one pip-installable package) can ship multiple plug-ins across categories — declare each as a separate entry-point.
Quickstart: hello-world event plug-in
This is the smallest end-to-end plug-in. Source lives under examples/mirror-plugin-echo/.
1. Create the package
mirror-plugin-echo/
├── pyproject.toml
└── mirror_plugin_echo/
└── __init__.py
pyproject.toml:
[project]
name = "mirror-plugin-echo"
version = "0.1.0"
dependencies = ["mirror.py>=1.0.0rc10"]
[project.entry-points."mirror.event"]
echo = "mirror_plugin_echo:plugin"
[build-system]
requires = ["setuptools>=84"]
build-backend = "setuptools.build_meta"
mirror_plugin_echo/__init__.py:
import logging
import mirror.event
import mirror.plugin
NAME = "echo"
def _on_status(package, status) -> None:
cfg = mirror.plugin.get_config(NAME)
prefix = cfg.get("prefix", "[echo]")
logging.getLogger("mirror").info(f"{prefix} {package.pkgid} -> {status}")
def setup() -> None:
mirror.event.on("MASTER.PACKAGE_STATUS_UPDATE.POST", _on_status)
def plugin():
from mirror.plugin import event_plugin
return event_plugin(name=NAME, setup=setup, api_version=(1, 0))
2. Install into the daemon’s environment
uv pip install ./mirror-plugin-echo
# or
pip install ./mirror-plugin-echo
3. Configure (optional)
/etc/mirror/config.json (the read-only-at-runtime config):
{
"settings": {
"plugins": {
"echo": {"enabled": true}
}
}
}
Per-plugin config lives in a separate file next to config.json. For the
echo plugin, create /etc/mirror/echo.json:
{"prefix": "[hello-from-echo]"}
If you skip both, the plug-in is enabled by default with empty config (prefix
falls back to [echo]).
4. Restart the daemon
For a systemd installation, restart the daemon and follow its journal to see the echo plug-in log each package status change:
sudo systemctl restart mirror.service
sudo journalctl -u mirror.service -f
File logs use the path configured in settings.logger.fileformat; see the
log layout.
Plug-in contract
Every plug-in module must expose a callable referenced by its entry point that returns a PluginRecord. Use the factory helpers — they validate the contract and produce the right shape.
sync plug-ins
from mirror.plugin import sync_plugin
def execute(package, pkg_logger):
"""Required. Drive the sync, eventually call mirror.sync.on_sync_done."""
...
def on_sync_done(package, pkg_logger, success, returncode):
"""Optional. Called when the worker reports completion."""
...
def setup():
"""Optional. Module-level init at registration time."""
...
def plugin():
return sync_plugin(name="myproto", execute=execute,
on_sync_done=on_sync_done, setup=setup)
execute() receives the Package and a per-sync Logger. It is called from a daemon thread. Common pattern: build a command, hand it to mirror.socket.worker.execute_command(...) for subprocess execution; the worker’s job_finished notification eventually calls mirror.sync.on_sync_done(pkgid, success, returncode) automatically. If your sync runs synchronously (no worker), call mirror.sync.on_sync_done yourself before returning — see the bundled local synctype for the simplest example.
event plug-ins
from mirror.plugin import event_plugin
def setup():
"""Required. Register listeners via mirror.event.on(...)."""
mirror.event.on("MASTER.PACKAGE_STATUS_UPDATE.POST", my_listener)
def plugin():
return event_plugin(name="my-notifier", setup=setup)
Available events:
Event name |
Args |
When it fires |
|---|---|---|
|
— |
Master daemon starting (before socket bind), |
|
— |
Master daemon ready, after socket bind |
|
|
Right before |
|
|
After mutation, after timestamps update |
|
— |
After a master-side restart re-establishes the worker connection |
Listeners run in a thread pool (ThreadPoolExecutor, 20 workers). If you mutate shared state, hold a lock. wait=True events block the caller until all listeners finish; wait=False events return immediately and listener exceptions are logged but not raised.
status plug-ins
Status plug-ins shape what mirror.py writes to stat.json and the web status
JSON. There are three modes — a single plug-in may use any combination.
Mode 1 — Extend (additive)
Add fields nested under a per-plug-in key. Multiple plug-ins coexist; namespacing prevents collisions.
from mirror.plugin import status_plugin
def extend_stat_fields(package) -> dict:
return {"my_metric": compute(package)}
def extend_web_status_fields(package) -> dict:
return {"freshness": classify(package.lastsync)}
def plugin():
return status_plugin(name="my-stats",
extend_stat_fields=extend_stat_fields,
extend_web_status_fields=extend_web_status_fields)
Contributions appear under statusinfo.plugins["my-stats"] and
web_status[pkgid]["plugins"]["my-stats"] respectively.
Mode 2 — Transform (replace fields)
Receive the entire payload mirror.py just built (extend hooks already applied) and return a transformed dict. Useful when the plug-in needs to restructure or replace fields, not just add them.
def transform_stat_payload(payload: dict) -> dict:
payload["mirrorname"] = payload["mirrorname"].upper()
return payload
def plugin():
return status_plugin(name="upper-name", transform_stat_payload=transform_stat_payload)
Single owner per channel. Each of transform_stat_payload and
transform_web_status_payload can be claimed by at most one plug-in. A
second registration raises ValueError at load time.
Mode 3 — Additional output files
Declare extra status files written alongside (not instead of) mirror.py’s
own outputs. Each output is single-owner by name.
from mirror.plugin import status_plugin, StatusOutput
def build_kaist_payload(packages):
return {
"timestamp": datetime.now(timezone(timedelta(hours=9))).isoformat(),
"package": {p.pkgid: kaist_shape(p) for p in packages},
}
def plugin():
return status_plugin(
name="kaist-status",
outputs=[
StatusOutput(
name="kaist-status",
default_path="/var/www/mirror/kaist-status.json",
build=build_kaist_payload,
config_path_key="output_path", # operator can override default_path via config
),
],
)
Operator can override the path through the per-plugin config file.
In config.json:
"plugins": {
"kaist-status": {"enabled": true}
}
In <config_dir>/kaist-status.json:
{"output_path": "/var/lib/mirror/kaist-status.json"}
config_path_key names the key ("output_path") that mirror.py reads from
the per-plugin file to override default_path.
Hook ordering and isolation
For each MASTER.PACKAGE_STATUS_UPDATE.POST event, mirror.py runs the
write pipeline in this order:
Build mirror.py’s default payload.
Apply each registered
extend_*_fieldshook (additive, namespaced).Apply the
transform_*_payloadowner if any (single owner, gets the already-extended dict).Atomic write to
stat.json(orstatus.jsonfor web).Iterate
outputsand atomic-write each.
Per-plug-in failures (transform raising, build raising, write raising) are
logged via mirror.log.warning and do not block sibling writes.
StatusOutput reference
Field |
Type |
Notes |
|---|---|---|
|
|
Globally unique across all plug-ins’ outputs |
|
|
Filesystem path (operator can override per-plug-in) |
|
|
Produces the JSON payload |
|
|
If set, plug-in’s |
Atomic writes
All status files (stat.json, status.json, plug-in outputs) are written via
tempfile + os.replace so readers never observe a partially-written file.
The default stat.json payload contains runtime state only and is written with
mode 0600. It no longer includes package configuration fields such as settings.
Stat transforms must consume runtime state rather than expecting configuration
in the input payload; extension hooks still receive the in-memory Package.
Plug-in status output files are written with mode 0644 so a web server or
other consumer running as a different user can read them.
Status plug-ins are trusted code: hooks, transforms, and output builders must
not copy credentials from a Package into their results.
API reference
mirror.plugin.PluginRecord
A frozen dataclass produced by the factory helpers. Fields:
Field |
Type |
Notes |
|---|---|---|
|
|
Globally unique across all categories |
|
|
|
|
`Callable |
None` |
|
`Callable |
None` |
|
`Callable |
None` |
|
callable or None |
Optional on-demand config-file creation callback |
|
string or None |
Per-plugin config filename; defaults to |
|
tuple of two integers or None |
Target core plugin API version |
|
`Callable |
None` |
|
`Callable |
None` |
|
`Callable |
None` |
|
`Callable |
None` |
|
`list[StatusOutput] |
None` |
You should not construct PluginRecord directly — always use the factory functions.
Factory functions
mirror.plugin.sync_plugin(name, execute, on_sync_done=None, setup=None,
create_config=None, config_filename=None, api_version=None) -> PluginRecord
mirror.plugin.event_plugin(name, setup, create_config=None,
config_filename=None, api_version=None) -> PluginRecord
mirror.plugin.status_plugin(name, extend_stat_fields=None, extend_web_status_fields=None,
transform_stat_payload=None, transform_web_status_payload=None,
outputs=None, setup=None, create_config=None,
config_filename=None, api_version=None) -> PluginRecord
Each validates its required arguments and raises TypeError on contract violation, so a plug-in author with a typo gets a clear error at import time.
API compatibility
External plugins should declare api_version=(1, 0) in their factory call.
The core exposes mirror.plugin.PLUGIN_API_VERSION, currently (1, 0).
A different major version or a newer minor version causes the plugin to be
skipped. Omitting the version still loads the plugin with a deprecation warning.
Declare the version your plugin was written against rather than copying the
running core version dynamically.
Creating plugin configuration
Plugins can provide create_config(force: bool) -> ConfigCreateResult. The
callback owns writing its default config and returns ConfigCreateResult(path, created). Preserve an existing file unless force=True. This callback runs
only when requested through mirror plugin config create NAME; it is not
called automatically during startup. The main config.json remains unchanged.
Use --config PATH to select the main config file and --force to request
overwriting an existing plugin config.
mirror.plugin.StatusOutput
A dataclass representing an additional output file written by a status plug-in.
Field |
Type |
Notes |
|---|---|---|
|
|
Globally unique across all plug-ins’ outputs |
|
|
Filesystem path (operator can override per-plug-in) |
|
|
Produces the JSON payload |
|
|
If set, plug-in’s |
Import via from mirror.plugin import StatusOutput.
Lookups
mirror.plugin.get_record(name) -> PluginRecord | None— registered record or None.mirror.plugin.get_config(name) -> dict— reads<config_dir>/<name>.json(or the file declared by the plugin’sconfig_filename). Returns{}if the file is missing or invalid. RaisesKeyErrorif the name is not registered.
Config schema
The plugins value under settings is an enable-only map of name -> {enabled: bool}:
"plugins": {
"echo": {"enabled": true},
"slack-notify": {"enabled": true},
"lftp": {"enabled": false}
}
A plug-in absent from the map defaults to enabled: true. Setting enabled: false
for a built-in (rsync, ftpsync, lftp, bandersnatch, local, ubuntu, jigdo, debmirror, apt-mirror2) prunes it from
mirror.sync.methods; subsequent package validation rejects packages that still
reference that synctype with ValueError("Sync type not in [...]").
Any config key left in config.json is now ignored (a warning is emitted at load
time) and should be moved to <name>.json in the same directory as config.json.
Per-plugin config files
Each plugin reads its own config from <config_dir>/<name>.json, where
<config_dir> is the directory containing config.json (typically /etc/mirror/).
The filename defaults to
<name>.jsonbut can be overridden by passingconfig_filename="myfile.json"to the factory helper (sync_plugin,event_plugin, orstatus_plugin) when declaring the plugin.mirror.plugin.get_config(name)returns{}if the file is missing or contains invalid JSON — it never raises from a bad operator file.The filename is restricted to a plain basename. Path traversal attempts (e.g.
../x.json, absolute paths) are rejected:get_configlogs a warning and returns{}.
The legacy list-of-strings shape ("plugins": ["/path/to/plugin.py", ...]) used by older mirror.py versions is detected at load time, logged as a deprecation warning, and ignored. There is no automatic migration — operators must rewrite the config to the dict shape.
Bootstrap order (why this matters for sync plug-ins)
Plug-in registration happens in two phases:
Phase A — built-ins, at package import time.
mirror/__init__.pycallsmirror.plugin.load_builtin_plugins(), which registers all nine built-in synctypes (rsync,ftpsync,lftp,bandersnatch,local,ubuntu,jigdo,debmirror,apt-mirror2). The module references are declared inmirror.plugin._BUILTIN_ENTRY_POINTSand published inpyproject.tomlunder themirror.syncentry-point group.Phase B — externals + disable, inside
mirror.config.load(). AfterConfig.load_from_dictparses the raw config but beforePackagesvalidates each package’s synctype, the loader:Removes built-ins that the operator disabled in config.
Iterates installed entry points across
mirror.sync/mirror.event/mirror.status, filtered to non-built-ins, and registers each (skipping ones the operator disabled).Calls
setup()on each newly registered plug-in.
This means when Package.from_dict checks synctype in mirror.sync.methods, the methods list reflects the union of built-ins (minus disabled) and externals (minus disabled).
event and status plug-ins do not affect package validation — they only need to be registered before whatever event or status write they hook.
Naming conventions
Distribution name: prefix with
mirror-plugin-(e.g.mirror-plugin-slack-notify). This is convention only; the loader does not enforce it.Internal
name: globally unique across all three categories. The loader keeps a single registry; a sync plug-in namedslackand an event plug-in namedslackcollide and the second registration raisesValueError.
Trust model
Plug-ins are arbitrary Python code that runs in-process at the daemon’s privilege level. There is no sandbox. The discovery channel is whatever wheels are installed in the daemon’s site-packages — same trust level as any other dependency.
Recommended:
Pin plug-in versions in your deployment manifest to avoid silent upgrades.
Audit plug-in source before installing, especially anything outside your organisation’s PyPI mirror.
Plug-in disable (
enabled: false) skipsep.load()entirely — the file is not even imported. Useful as a kill switch.
Future: pluggy adoption
The entry-point group names (mirror.sync, mirror.event, mirror.status) were chosen to be compatible with pluggy. When/if mirror.py adopts pluggy hookspec/hookimpl, already-published plug-ins continue to work — pluggy’s pm.load_setuptools_entrypoints("mirror.<group>") consumes the same entry-points. Plug-in authors would only need to add @hookimpl decorators to their existing functions.
This is currently out-of-scope for the codebase but informs the design.
Worked example
See examples/mirror-plugin-echo/ for a complete, installable event plug-in package. It logs every package status change to mirror’s logger with a configurable prefix.