CelebiOpen source · Apache-2.0
Reproducible analysis management for high-energy physics
Celebi organizes projects, data, algorithms, and tasks into a structured workspace — with dependency tracking, provenance, and snapshotting built in — so complex HEP analysis chains stay reproducible, traceable, and collaborative.
Core concepts
A small vocabulary that keeps large analyses organized.
Architecture
A code and metadata repository, plus a production factory that executes workflows.
Repository
An organized directory of objects, each carrying machine and human metadata.
Materialized workflow
Every workflow node is a real folder on disk — not an abstract step in a description file.
Impression
Immutable, content-defined snapshots of every workflow node.
Yuki
The middleware (DITE) that queues operations, dispatches jobs, and owns the data.
Runner
REANA, local, or ssh — interchangeable backends that execute impressions.
Features
Structured organization
Clear separation of projects, data, algorithms, and tasks within a well-defined hierarchy.
Dependency tracking
Relationships between data, algorithms, and tasks form a directed acyclic graph you can inspect and replay.
Impressions
Snapshot important results, configurations, and object states over time — a built-in versioning memory.
Reproducibility
Workflow structure, parameters, inputs, and execution environments are captured completely.
Adaptability
Change an algorithm or a parameter and re-run only the affected downstream tasks.
Collaboration
Share projects and workflows consistently across people, machines, and environments.
Examples
Example analyses built with Celebi — more coming soon.
Beginner
Hello world workflow
A minimal two-task chain: create an algorithm, wire a task, submit, and seal your first impression.
Intermediate
Template fitting
A template-fit analysis with upstream selection tasks — demonstrates dependency tracking and selective re-running.
Advanced
Systematics scan
One algorithm, many parameter variations — shows how tasks fan out and how impressions capture each state.
News & Roadmap
General
- A Novel Architecture for Reproducible and Long-Term Preserved Physics Analyses
- Desktop UI: DAG viewer, runner management, and job status
- CLI: project-root @/ paths with tab completion
- A Novel Architecture for Reproducible and Long-Term Preserved Physics Analyses
- VS Code extension: styled output and README rendering
- Celebi skills for Claude Code released
Releases
- Clear entry points: `celebi` for the shell, `celebi-cli` for commands, and `celebi-git` for Git
- Three help forms: `helpme`, `help <command>`, and `<command> --help`
- Lifecycle-aware, idempotent submission with guarded retries
- `register-data` registers data that already lives on an SSH runner — MD5 computed remotely, zero network transfer
- Data commands renamed to directional verbs: `upload-data`, `attach-data`
- `cache_on_runner` task option (replaces `use_eos`): EOS on REANA, runner impressions on SSH
- Project-root-relative `@/` paths in the shell, with tab completion
- Selective `impress a b` / `submit a b` by object names; `submit --runner <name>`
- `workaround --reference <algorithm>` and `--skip-input <task>`
Roadmap
Current
- CelebiChrono core: projects, tasks, algorithms, data, impressions
- CLI with project-root @/ path support and tab completion
- VS Code extension: tree view, submit, trace visualization
- Desktop UI: DAG viewer, runner management, integrated terminal
Near-term
- PyPI packaging and streamlined installation
- Batch and remote runners (HTCondor, Slurm)
- Expanded documentation and tutorials
- Web-based monitoring dashboard
Future
- Collaborative project registry and sharing hub
- Cloud execution backends
- Experiment-specific integrations and templates