experiments package

experiments.slurm_fetchback module

Fetch result data back from cluster to local hard drive.

Instructions:

  • Define scenarios/parameter sets in settings_local.toml

  • execute main

Authors:

fetch_outputfiles_from_cluster(run_id: int = None)[source]

Fetch output files from cluster via SCP

Other Parameters:
- settings.slurm.fetch_pattern
- settings.slurm.output_path
- settings.slurm.host
- settings.slurm.username
- settings.merge_output.run_ids

experiments.slurm_management module

Create bash scripts for simulation on SLURM managed cluster, transfer these scripts to the cluster and optionally execute them.

Instructions:

  • Define scenarios/parameter sets in excel scenario sheet, referenced by config ID

  • execute main

Authors:

transferSlurmFiles(maxRunId)[source]

Transfer files to cluster via SCP

Parameters:
maxRunID: int

maximum run ID of this job set

transferSettingsFile()[source]

Transfer settings.toml

transferScenarioFile()[source]

Transfer Scenario Config Excel File

executeSlurmScript(execScriptFilename, maxRunId)[source]

Execute slurm batch runs on cluster via SSH

Parameters:
execScriptFilename: str

Script file to execute

maxRunId: int

maximum Run ID to enter correct folder

generateAndTransfer()[source]

Generates required files, transfers these to the cluster and executes the SLURM script on the cluster. In particular:

  • generate SLURM scripts

  • transfer SLURM scripts to cluster

  • transfer settings file to cluster

  • transfer scenario file to cluster

  • execute SLURM scripts on cluster

Particular steps can be enabled/disabled by parameter settings (see below).

Considered parameters:

  • settings.slurm.batchconfig_file

  • settings.slurm.batchconfig_sheetname

  • settings.slurm.batchconfig_rows

  • settings.slurm.editbatchConfigFile (default: True)

  • settings.slurm.generateSlurmFile (default: True)

  • settings.slurm.transferSlurmFile (default: True)

  • settings.slurm.transferSettingsFile (default: True)

  • settings.slurm.transferScenarioExcelFile (default: True)

  • settings.slurm.executeSLURMscripts (default: False)

experiments.slurm_script_generation module

Create SLURM scripts to run AHOI on a linux cluster. Creates multiple runs depending an seed range defined in scenario file (per row) and subsample sets.

Authors:

generate_script(args=['source', '../_build'])[source]

Creates SLURM scripts by replacing placeholders in template and generates execution script to call SLURM scripts.

Parameters:
run_id: int

Run ID

args: list[str]

arguments for SLURM script creation

Returns:
str

filename of execution script

Other Parameters:
* settings.slurm.template_file
* settings.slurm.target_batchfiles
* settings.slurm.target_execfile
* settings.slurm.target_cluster_logfiles
* settings.slurm.target_cluster_batchfiles

Module contents