Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
sparkctl documentation
sparkctl documentation

Contents:

  • How Tos
    • Getting Started
      • Installation
      • Deploy sparkctl in an HPC environment
    • Configuration
      • How to Select Compute Nodes
      • Heterogeneous Slurm jobs
      • How to handle compute node failures
      • How to use a custom spark-default.conf file
      • How to Set a Custom Spark Log Level
    • Execution
      • Start a Spark Cluster
      • How to run Spark jobs in Python
      • How to to monitor Spark resource utilization
    • Applications
      • How to configure a Hive metastore
      • Visualize Data with Tableau
    • Debugging
  • Tutorials
    • Run jobs in a Python script
    • Run jobs interactively with sparkctl
    • Use Ibis with sparkctl
    • Run jobs with Spark Connect
    • Run jobs with spark-submit or pyspark
  • Reference
    • sparkctl API
    • HPC
      • NLR Kestrel
    • CLI Reference
  • Explanation
  • Frequently Asked Questions
Back to top
View this page

HPCΒΆ

This section contains how-tos for specific HPC environments.

Contents:

  • NLR Kestrel
    • Spark software
    • Compute nodes
Next
NLR Kestrel
Previous
sparkctl API
Copyright © 2026, Alliance for Energy Innovation, LLC
Made with Sphinx and @pradyunsg's Furo