Warning generated about logged columns

[tbhallett]

When running a Scenario, I see the following warning issued. I presume it's of interest right now to @matt-graham and is related to #1434 and #1435

/Users/tbh03/GitHub/TLOmodel/src/tlo/scenario.py:438: InconsistentLoggedColumnsWarning: Inconsistent columns in logged values for tlo.scenario logger with key override_parameter compared to header generated from initial log entry:
  Columns in header not in logged values are
  {'new_value': 'bool', 'old_value': 'bool'}
  Columns in logged values not in header are
  {'new_value': 'str', 'old_value': 'str'}
  logger.info(

[matt-graham]

Hi @tbhallett. Thanks for flagging this.

I don't think this is directly related to #1434 and #1435 but the warning being generated is due to the changes in #1404 which is part of where #1434 arose from!

We now generate a warning if we create structured log entries where the 'columns' of the record differ between the record used to generate the header (first record for a given key) and subsequent records, where the difference can be in the column names or associated types. Here it looks like the first record for the override_parameter key in the tlo.scenario log had both new_value and old_value columns with type bool while a subsequent log record had str type for these columns. Looking at where this log entry is generated

TLOmodel/src/tlo/scenario.py

Lines 426 to 447 in 9561a7a

	for module_name, overrides in overridden_params.items():
	if module_name in sim.modules:
	module = sim.modules[module_name]
	for param_name, param_val in overrides.items():
	assert param_name in module.PARAMETERS, f"{module} does not have parameter '{param_name}'"
	# assert np.isscalar(param_val),
	# f"Parameter value '{param_val}' is not scalar type (float, int, str)"
	old_value = module.parameters[param_name]
	assert type(old_value) is type(param_val), f"Cannot override parameter '{param_name}' - wrong type"
	module.parameters[param_name] = param_val
	logger.info(
	key="override_parameter",
	data={
	"module": module_name,
	"name": param_name,
	"old_value": old_value,
	"new_value": module.parameters[param_name]
	}
	)

this is probably as the overriden parameters in the overrides dictionary which is looped over here are not necessarily of consistent types, so this is technically not a valid way to log them (as we assume all entries for a given columns are all of the same type).

To resolve this, we could either convert all entries for old_value and new_value to their string representations by passing to str or create a data structure outside of the nested loops which is then logged in one go.