Configuration

Configuration and parameter management is done via the Dynaconf Framework.

Warning

Because of bug #896 in dynaconf (and because we apply merge_enabled=True) it is currently not straight forward to prevent the merging of lists in the configuration, i.e. items of a list defined in settings_default.toml will be kept even if the setting is overridden in other places. Therefore, defaults of list settings which may be overriden should be initialised empty. A workaround is using CLEAR to override list entries as the first entry of the new list. The according setting key needs to be added to config/dynaconf_hooks.py.

Order of configuration consideration

Default values for all settings are meant to be defined in settings/settings.toml in the repos. Your personal settings that might override the default ones usually go into settings/settings_local.toml.

1. Configuration in settings.toml

2. Configuration in settings_local.toml (or whatever is set in env var AHOI_SETTINGS_FILE_FOR_DYNACONF)

3. Configuration in .secrets.toml

4. Configuration in constants.toml

5. Configuration in Scenario XLSX file (if MAIN__EXCEL_SCENARIO_FILE is set)

6. Environment variables

Warning

Note that settings integrated by includes are not processed right after the file they are defined in but after settings in constants.toml and all loaders. Special attention must be paid to settings which affect other loaders, e.g. MAIN__EXCEL_SCENARIO_FILE and AHOI_MAIN__SCENARIO_ID. Defining these as includes usually comes to late!

Background

Environment are off because we use sections to cluster parameters. Global merging is on because we want to combine parameters within parameter clusters.

Hint

You can apply settings for specific environments/context such as the cluster by passing the according settings filename (absolute or relative to building-stock-model/building_stock_model/config) to the environment variable AHOI_SETTINGS_FILE_FOR_DYNACONF.

Defining Scenarios

Often, many runs need to be conducted with changes in only one or few parameters. In this case it is convenient to fill an excel table with lines for each particular scenario parameter set, indexed by a so-called scenario ID. The scenario ID (MAIN__SCENARIO_ID) can then be switched via an environment variable, e.g. in the SLURM configuration file when running on a cluster or in the personal settings.toml file.

The Scenario excel file looks like this (there is a template input_data/Scenario_Template.xlsx):

Scenario_Template.xlsx

	MAIN	MAIN	MAIN	RNG	RNG
ScenarioID	STARTYEAR	STEPSIZE	FINALYEAR	seed_buildinginit	seed_buildingnew
1	2020	5	2050	1	1
2	2020	1	2050	2	2
3	2022	5	2050	1	1
4	2022	1	2050	2	2

The part of the SLURM script may look like this:

Part of SLURM config file

 echo "started..."
 export AHOI_MAIN__SCENARIO_ID=1
 export AHOI_OUTOUT_OUTPUT_SETTINGS=TRUE
 srun python ./src/run.py
 echo "finished"

The excel file is specified via MAIN__EXCEL_SCENARIO_FILE (relative to MAIN__INPUT_PATH or absolute). To disable loading settings from excel file, set

Disable loading settings from excel file

 [main]
 excel_scenario_file = "@none"

Conserving Scenario Settings

If OUTPUT__OUTPUT_SETTINGS is True (the default), applied settings are exported to <OUTPUT__OUTPUT_PATH>/<OUTPUT__SUBFOLDER>/<OUTPUT__OUTPUT_SETTINGS_PATH>/settings.toml

Config for output of applied settings

 [output]
 output_settings = true
     output_settings_path = "conf"
     output_settings_filename = "settings.toml"

Warning

Make sure to store the excel files and other input data which is referenced in the stored settings.toml as they are during simulation!

Inspecting Settings History

In case of application of unexpected settings it can be helpful to inspect the history of settings. Simply make sure the switch OUTPUT__OUTPUT_SETTINGS is True (default) and optionally filter by a settings key (part) for OUTPUT__OUTPUT_SETTINGSHISTORY_KEY:

Config for output of settings history

 [output]
 output_settings = true
 output_settingshistory_filename = "settings_history.json"
 output_settingshistory_key = "@none"

The according dynconf feature is documented as Inspecting History. It shows first the finally applied settings and afterwards exports the history with the according loader and its parameters from last to first.

Settings Documentation

main

Setting	Unit	Type	Default	Scope	Description	Annotations
current_scenario			“Scenario_heat_pumps”
project			“AHOI”
run_id			“0”
task			“”
start_year			2025
end_year			2045
number_of_plumbers			10
number_of_energy_advisors			10
steps			1040
serialization_function			“to_excel”
files_prefix			“”
input_path			“./data/input/”
output_path			“./src/data/output”			Output ‘root’ for logs, settings history, graphml files etc. Absolute or relative to working dir.
excel_scenario_file			“settings/settings.xlsx”
config_id_start			0
config_id_end			0

experiments

Setting	Unit	Type	Default	Scope	Description	Annotations
plumber_training			false
plumber_training_system			“Heating_system_heat_pump_brine”
information_campaign			false
inf_campaign_systems			[]
inf_campaign_duration			[]
inf_campaign_reach			0
inf_campaign_mode			“Risk_targeting”
inf_campaign_risk_mitigation			0
enforcement			false
enforcement_systems			[]
replacement_mandates			false
emissions_mandate			1000000000
systems_mandate			[]

data

Setting	Unit	Type	Default	Scope	Description	Annotations
dynamic			false
heat_demand			“eqcity”
milieu_params			“milieu_parameters.csv”
heat_sys_params			“heating_parameters.csv”
heat_sys_params_dynamics			“heating_params_dynamics.csv”
plt_settings			“plotting/config_plt_unik.yaml”

scenario_comparison

Setting	Unit	Type	Default	Scope	Description	Annotations
active			false
scenarios			[]
files_prefixes			[]
run_ids			[]

seeds

Setting	Unit	Type	Default	Scope	Description	Annotations
model_init			0
model_run			0
house_init			0
milieu_init			0
heating_init			0
information_source_run			0
houseowner_run			0
plumber_run			0

output

Setting	Unit	Type	Default	Scope	Description	Annotations
randomstreaminvocations			false
randomstreaminvocations_folder			“@format ./data/output/rng/{this.main.run_id}/”
randomstream_cutentries			2
logger_model_init			“test_model_init_”
logger_model_run			“test_model_run_”
logger_house_init			“test_house_init_”
logger_milieu_init			“test_milieu_init_”
logger_houseowner_run			“test_houseowner_run_”
logger_plumber_run			“test_plumber_run_”
logger_information_source_run			“test_information_source_run_”
logger_heating_init			“test_heating_init_”
resultdata_as_csv			false
agentdata_as_pickle			true
intermediarydata_as_pickle			true
simplenames			false
output_settings			true
output_settings_path			“conf”
output_settings_filename			“settings.toml”
output_settingshistory_filename			“settings_history.json”
output_settingshistory_key			“@none”
hdf5_filename			“AHOI.h5”
hdf5_format			“fixed”

information_source

Setting	Unit	Type	Default	Scope	Description	Annotations
distortion			0.5
uncertainty_lower			0.2
uncertainty_upper			0.5

subsidies

Setting	Unit	Type	Default	Scope	Description	Annotations
district			0.3
heat_pump			0.3
heat_pump_brine			0.3
pellet			0.3
network_local_hot			0.3
GP_Joule			0.3
climate_speed			0.2
income			0.3
efficiency			0.05

houseowner

Setting	Unit	Type	Default	Scope	Description	Annotations
aspiration			1
overload			2
perception			0.1
budget_limit			104
risk_tolerance_std			0.19
random_risk_tolerance			true
loan_taking_probability			0.32
tpb_weights_std			[]			xactly three numbers

houseowner.leading

Setting	Unit	Type	Default	Scope	Description	Annotations
mean_savings			178
stdev_savings			66
risk_tolerance			1

houseowner.mainstream

Setting	Unit	Type	Default	Scope	Description	Annotations
mean_savings			142
stdev_savings			62
risk_tolerance			0.7

houseowner.traditionals

Setting	Unit	Type	Default	Scope	Description	Annotations
mean_savings			158
stdev_savings			69
risk_tolerance			0.5

houseowner.hedonists

Setting	Unit	Type	Default	Scope	Description	Annotations
mean_savings			158
stdev_savings			58
risk_tolerance			0.6

information_overspread

Setting	Unit	Type	Default	Scope	Description	Annotations
selected_optio			“subsidy_to_observe”			choices: subsidy_to_observe, known_hs_to_observe
subsidies_list			[“Climate_speed”, “Heat_pump”, “Income”]
subsidies_current			“Climate_speed”
known_hs_list			[
known_hs_current			“Heating_system_heat_pump”

decision_making_costs

Setting	Unit	Type	Default	Scope	Description	Annotations
evaluate			1
get_data			2
define_choice			1
compare_hs			1
install			1
calculate_satisfaction			1

intermediary

Setting	Unit	Type	Default	Scope	Description	Annotations
job_duration			1

plumber

Setting	Unit	Type	Default	Scope	Description	Annotations
aspiration			100
perception			0.1
cognitive_resource			20
hs_budget			200
cons_duration			1
ins_duration			1
share_systems			true
assume_area_avg			106
assume_energydemand_avg			147
assume_heatload_avg			19

energy_advisor

Setting	Unit	Type	Default	Scope	Description	Annotations
cons_duration			1

heating_systems_colors

Setting	Unit	Type	Default	Scope	Description	Annotations
heating_system_electricity			“red”
heating_system_gas			“blue”
heating_system_heat_pump			“yellow”
heating_system_heat_pump_brine			“purple”
heating_system_oil			“black”
heating_system_pellet			“brown”
heating_system_network_district			“pink”
heating_system_network_local			“orange”

reference.Scenario_heat_pumps

Setting	Unit	Type	Default	Scope	Description	Annotations
Heating_system_heat_pump			100

reference.Scenario_network_district

Setting	Unit	Type	Default	Scope	Description	Annotations
Heating_system_network_district			100

reference.Scenario_network_local_hot

Setting	Unit	Type	Default	Scope	Description	Annotations
Heating_system_network_local			100

reference.Scenario_network_local_cold

Setting	Unit	Type	Default	Scope	Description	Annotations
Heating_system_heat_pump_brine			100

reference.Scenario_mix_pellet_heat_pump

Setting	Unit	Type	Default	Scope	Description	Annotations
Heating_system_heat_pump			80
Heating_system_pellet			20

reference.Scenario_mix_pellet_heat_pump_network

Setting	Unit	Type	Default	Scope	Description	Annotations
Heating_system_heat_pump			10
Heating_system_pellet			30
Heating_system_network_local			60

information_sources

Setting	Unit	Type	Default	Scope	Description	Annotations
list			[“internet”, “magazine”, “plumber”, “neighbours”, “energy_advisor”]

network

Setting	Unit	Type	Default	Scope	Description	Annotations
scenario_id			0

milieu

Setting	Unit	Type	Default	Scope	Description	Annotations
groups			[“Generalized”, “Leading”, “Mainstream”, “Traditionals”, “Hedonists”]

triggers

Setting	Unit	Type	Default	Scope	Description	Annotations
adoptive_trigger			“Heating_system_heat_pump”
adoptive_trigger_active			false

slurm

Setting	Unit	Type	Default	Scope	Description	Annotations
template_file			“src/experiments/slurm_script_template.sh”			relative to get_input_path:
target_execfile			“slurm”			relative to get_batchconfig_path:
target_batchfiles			“slurm/batchfiles”			relative to get_batchconfig_path (should be subfolder of target_execfile):
target_cluster_mainpath			“~/ahoi/”			project root path on cluster
target_cluster_modelbase			“@format {this.slurm.target_cluster_mainpath}/ahois-pro/src/”			path to agenthomid code at cluster
target_cluster_batchfiles			“./batchfiles”			relative to target_excefiles
target_cluster_logfiles			“@format {this.slurm.target_cluster_mainpath}/{this.main.task}”			path to store slurm log files at cluster (needs to correspond with the pattern defined in template_file - the part before the RunID / Job which are added programmaatically):
host			“its-cs1.its.uni-kassel.de”
username			“uk093483”
num_runs_per_batch			1
config_id_start			0
config_id_end			0
fetch_pattern			“pickles/.”
partitio			“FB16”
duratio			300

eval

Setting	Unit	Type	Default	Scope	Description	Annotations
language			[“en_GB”]			anguage = [“de_DE”]
colname_heating			“Heizsystem”
ahid_settings_files			[“ahid_settings_default.toml”]
process_agent_files			true
target_samplesize			1000
start_year			2025
target_year			2045
plot_title			false
xticks			[2025, 2030, 2035, 2040, 2045]
years_in_period			5
widt			12
widthfactorbox			0.6
height			7
dpi			192
resolution			192
markevery			52
markersize			10
labelrotation			30
labe			“NN”
use_adjust_text			false
show_legend			true
process_all_outputs			true
analyze_obstacles			false
analyze_obstacles_by_period			false
individual_plots_by_ids			false
labels_file			“settings/figure_labels.xlsx”
compare_fulfillment			true
compare_emissions			true
compare_energy_demand			true
compare_optimality			true
compare_opex			true
compare_total_expenses			true
compare_total_effort			true
compare_heating_systems_distribution			true
compare_hs_knowledge			true
compare_obstacles_counts			true
compare_obstacles_percentage			true
compare_attributes			true
title_fulfillment			true
title_hs_distribution_final			true
title_emissions			true
title_quota			true
xlabel_hs_distribution_final			true
xlabel_obstacles_percentage			true

slides

Setting	Unit	Type	Default	Scope	Description	Annotations
config			“AHOI_SlideGeneration.xlsx”
template			“/plotting/DT2_Slides_Template_Interventions.pptx”
sections			true
section_title_variants			false
font			“Frutiger LT 45 Light”
fontface_bold			true
fontsize_pt			6

Create settings documentation

Hint

CSV files can be created automatically for new settings (sections) via the script doc/source/application/concepts/settings/parse_settings2doc.py. Comment lines above the setting are used in the Annotations column. To add unit, type, scope, description and annotation values, use LibreOffice and store with “_edit” added. Use then the above mentioned script to convert line endings in multiline cells. Adapt the filename of edited CSV files in settings_doc.rst.