A prototype for interactive analysis
This project is a prototype for a data analysis system, CMS compliant. The main targets are:
- Reducing analysis "time to insight" (training time for newcomers included) with an interactive and user-friendly UI
- Single and easily accessible hub to reduce the complexity and maintenance of multiple and slightly overlapping solutions
- Increasing the system delivered throughput (
evts/s
)
Current design
The environment is composed using:
- JupyterHub and JupyterLab to manage the user interaction part of the infrastructure
- Dask to introduce the scaling over a batch system
- XRootD as data access protocol toward AAA
At the moment, it supports scaling over HTCondor clusters using a custom dask-jobqueue module.