Dask-CHTC builds on top of Dask-Jobqueue to spawn Dask workers in the CHTC HTCondor pool. It also provides tools for running Jupyter notebook servers in a controlled way on CHTC submit nodes, which you may find helpful for providing an interactive development environment to use Dask in.
If you’re interested in using Dask at CHTC but have never used CHTC resources before, please fill out the CHTC contact form to get in touch with our Research Computing Facilitators. If you’ve already had an engagement meeting, send an email to firstname.lastname@example.org and let them know you’re interested in using Dask.
We currently only support the Dask-CHTC workflow on the
submit3.chtc.wisc.edu submit node.
If you do not have an account on
submit3.chtc.wisc.edu, you will need to
Dask-CHTC is prototype software! If you notice any issues or have any suggestions for improvements, please write up a GitHub issue detailing the problem or proposal. We also recommend “watching” the GitHub repository to keep track of new releases, and upgrading promptly when they occur.
These pages will get you started with Dask-CHTC:
- Installing Dask-CHTC
How to install Python and Dask-CHTC on a CHTC submit node.
- Running Jupyter through Dask-CHTC
How to use Dask-CHTC to run a Jupyter notebook server on a CHTC submit node.
- Networking and Port Forwarding
Information on CHTC networking and how to forward ports over SSH, which will allow you to connect to Jupyter notebooks and Dask dashboards running on CHTC submit nodes.
- Dask Cluster Creation
A brief example Jupyter notebook, showing how to start up a
CHTCClusterand use it to perform some calculations.
These pages have information for troubleshooting problems and handling specific use cases:
Solutions and advice for tackling specific problems that might arise while using Dask-CHTC.
Information on how to configure Dask-CHTC and the Dask JupyterLab extension.
- Building Docker Images for Dask-CHTC
Information on how to build Docker images for use with Dask-CHTC.
Detailed information on the Python API and the associated command line tool can be found on these pages:
- API Reference
API documentation for
- CLI Reference
Documentation for the