API reference¶
This reference is generated from the source docstrings. See Authoring tools in dosho for the design behind these modules.
Top-level package¶
dosho – a shinobi’s tool bag. The native shinobi (stimela-ninja) cab repository. See AGENTS.md for the design rationale.
- dosho.define_cab(name, command, image, fields, *, outputs=None, field_meta=None, policies=None, input_patterns=None, output_patterns=None, flavour='binary', wranglers=None, info=None)[source]¶
Build a Cab from a flat {raw_param_name: (dtype, required, default)} dict – the shape a tool’s own –help/docs naturally give you (e.g. “data-ms”: (“MS”, True, None)). Hyphenated/dotted raw names are sanitised to valid pydantic field names, with the original kept as a nom_de_guerre so the built cab’s build_argv still emits the tool’s real flag.
field_meta entries the caller passes explicitly (keyed by the sanitised field name – e.g. an output field’s implicit template, or a positional/repeat_as_tokens override) are merged in on top of the auto-derived nom_de_guerre`s; an explicit entry that doesn’t set its own `nom_de_guerre keeps the auto-derived one rather than losing it.
- Parameters:
name (str)
command (str)
image (str)
policies (Policies | None)
input_patterns (list[ParamPattern] | None)
output_patterns (list[ParamPattern] | None)
flavour (str)
info (str | None)
- Return type:
Registry (string-keyed lookup)¶
name -> Cab/StepRef map, registered under shinobi’s shinobi.cabs entry-point group (see this package’s pyproject.toml). String-keyed runtime lookup, for a caller that doesn’t know the tool name until it runs (ninja cabs list/show, shinobi.cabs entry-point discovery) – for the write-time-known case, from dosho.cabs import <tool> (see dosho/cabs/__init__.py) is the more ergonomic, direct interface.
Tools¶
Every Cab/pystep dosho ports, re-exported at package level – see
Authoring tools in dosho for the single-command vs. multi-command
module convention. Individual tool modules aren’t documented
one-by-one here (their own docstrings explain each tool’s provenance and
any real-tool quirks preserved); import or inspect them directly, e.g.:
from dosho.cabs import wsclean
from dosho.cabs.casatasks import listobs
$ ninja cabs show wsclean
Every tool dosho ports, re-exported by name at package level, so a pipeline author who knows what they want at write-time can do from dosho.cabs import wsclean or from dosho.cabs.casatasks import listobs directly – no need to know or care whether a given tool is a Cab (a real binary, argv-built and shelled out to) or a StepRef (a Python-package tool with no standalone binary, run via @shinobi.pystep and ctx.import_func inside a container instead – CASA tasks are the only example so far). Both shapes are first-class for Recipe.add_step.
For runtime/string-keyed lookup (the CLI, shinobi.cabs entry-point discovery), use dosho.get(name)/dosho.registry instead – that’s a second, parallel interface for a caller that doesn’t know the name until it runs, not a replacement for these direct imports.
Importing this module (or any of its submodules – Python always initializes a parent package first) eagerly constructs every Cab/ StepRef below. That’s a deliberate, accepted tradeoff here: each one is cheap (pydantic model + dict construction, no I/O), so the “don’t pay for what you don’t use” laziness dosho.registry’s own submodule-path table still gives a caller that only wants one specific tool is preserved there, not here.
Shinobi schema types¶
Supporting types used when defining a Cab, re-exported from
shinobi for convenience – documented in full in shinobi’s own API
reference.
- class shinobi.steps.schema.ParamMeta(*, nom_de_guerre=None, implicit=None, info=None, positional=False, repeat_as_tokens=False, dtype=None)[source]
Bases:
BaseModelPer-field metadata a plain pydantic model can’t express: the name the underlying tool actually expects (nom_de_guerre), a value always supplied by the cab itself rather than the caller (implicit), human-facing help (info), the cab dtype string (dtype, e.g. “File”/”MS”) for a ParamPattern attr – since a dynamically-named param has no declared field/type annotation for path_fields to inspect, this is how backends know to bind-mount its directory – positional: emitted as a bare value (no –flag), in field-declaration order, after every flagged/pattern-matched arg – and repeat_as_tokens: a list/tuple value is emitted as separate bare argv tokens (after the one flag occurrence, or as separate positional tokens) instead of joined into one comma-separated token – real cult-cargo cabs express this as a per-field policies: {repeat: list} (see e.g. wsclean’s -size <w> <h>/-weight briggs <n>, which need two separate argv tokens, not “4096,4096” as one).
On an output field, a string implicit containing {name} placeholders is resolved by steps.dispatch._fill_outputs as a str.format template against the step’s prepared (validated) input values – e.g. implicit=”{prefix}-MFS-image.fits” derives a tool’s output path from its own prefix input, without shinobi ever importing/executing the tool’s own schema-generation code. A plain string with no {…} is used as a literal constant, same as on an input field.
- Parameters:
- implicit: Any
- positional: bool
- repeat_as_tokens: bool
- model_config = {}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class shinobi.steps.schema.ParamPattern(*, separator='.', segments)[source]
Bases:
BaseModelA family of inputs whose names are <segment><separator><segment>…, e.g. QuartiCal’s K.type/G.time_interval or cubical’s g1-solvable/ g-time-int. Matched as one anchored regex assembled from segments: exactly one segment is attrs (the known, enumerable part, each value with its own ParamMeta – dtype/nom_de_guerre/info); every other segment is a regex (soft shape-validation of a level that can’t be enumerated ahead of time). See AGENTS.md for the motivating tools.
The attrs segment is usually last (cubical/QuartiCal: an unenumerable term name followed by a known attribute, g1.solvable), but doesn’t have to be – wsclean’s dynamic output names are the opposite shape, a known/enumerable image type followed by an open-ended qualifier tail (dirty.per-band, restored.i.per-interval.mfs), so attrs there is the first segment. Only one segment may carry attrs; the rest must all be regex.
A segment regex that should behave as an unconstrained “match anything” level (the old design’s prefix) should be written lazily (.+?, not .+): with more than one registered attr, an eager .+ prefers the shortest attr that completes an overall match, which is wrong when one attr is itself a suffix of another (e.g. “int” vs “time-int” with separator “-”) – .+? tries the shortest prefix first, which is exactly “prefer the longest/most specific attr”.
- Parameters:
separator (str)
segments (list[ParamSegment])
- separator: str
- segments: list[ParamSegment]
- model_config = {}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_post_init(context, /)
This function is meant to behave like a BaseModel method to initialize private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (BaseModel) – The BaseModel instance.
context (Any) – The context.
- Return type:
None
- class shinobi.steps.schema.ParamSegment(*, regex=None, attrs=None)[source]
Bases:
BaseModelOne level of a dotted/dashed dynamic-parameter name. A “shape” segment carries only regex – soft validation, no metadata, for a level whose actual values can’t be enumerated at cab-authoring time (e.g. a solver term name like QuartiCal’s K/G). The “meta” segment – always the last one in a ParamPattern – carries attrs: the known, enumerable part, each value with its own ParamMeta.
- model_config = {}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class shinobi.steps.schema.Policies(*, prefix='--', replace=<factory>, list_sep=', ', repeat_list=False, key_value=False, repeat=None, explicit_true=False, explicit_false=False)[source]
Bases:
BaseModelHow a cab’s parameters are turned into command-line arguments.
key_value/repeat mirror real cult-cargo cab-level policy keys verbatim (e.g. QuartiCal’s policies: {key_value: true, repeat: ‘[]’, prefix: ‘’}): key_value=True means a hydra-style single name=value argv token instead of two tokens (–name, value); repeat=”[]” means a list value formats as one bracketed-literal token (solver.terms=[K,G]) instead of list_sep-joining. Distinct from a per-field ParamMeta.repeat_as_tokens (real per-field policies: {repeat: list}, e.g. wsclean’s bare -size 4096 4096), which is a field-level override and takes precedence when set.
explicit_true/explicit_false also mirror real cult-cargo cab-level policy keys verbatim (e.g. CubiCal’s policies: {explicit_true: true, explicit_false: false}): by default a True boolean value emits as a bare flag (–flag, argparse store_true-style) and False is omitted entirely. Some real CLIs (CubiCal’s own optparse-derived parser among them) instead expect every boolean option to always take an explicit value token – passing a bare flag with no value corrupts parsing of everything after it, since the parser consumes the next token as that flag’s value. explicit_true=True emits –flag true (two tokens, “true”/”false” lowercase) instead of a bare flag when the value is True; explicit_false=True does the same instead of omitting the flag when the value is False. Each direction is independent (CubiCal only needs explicit_true, never explicit_false), and this applies uniformly to declared fields and ParamPattern-matched dynamic ones (e.g. CubiCal’s own per-Jones-term g-solvable).
- Parameters:
- prefix: str
- list_sep: str
- repeat_list: bool
- key_value: bool
- explicit_true: bool
- explicit_false: bool
- model_config = {}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].