2025-06-13 13:48:55.421 | INFO | climate_ref_core.logging:redirect_logs:195 - Running definition ExecutionDefinition( diagnostic=ENSOCharacteristics(name='ENSO Characteristics'), key='cmip6_gn_r1i1p1f1_NorESM2-LM', datasets=, output_directory=PosixPath('/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09'), _root_directory=PosixPath('/nird/scratch/yanchun/diagnostics/climate-ref/scratch') ) 2025-06-13 13:48:55.462 | INFO | climate_ref_core.providers:create_env:346 - Environment at /nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d already exists, skipping. 2025-06-13 13:48:55.462 | INFO | climate_ref_core.providers:run:402 - Running '/nird/datalake/NS16000B/climate-ref-software/conda/micromamba run --prefix /nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d esmvaltool run --config-dir=/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/config /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/recipe.yml' 2025-06-13 13:49:01.849 | ERROR | climate_ref_core.providers:run:416 - Failed to run ['/nird/datalake/NS16000B/climate-ref-software/conda/micromamba', 'run', '--prefix', '/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d', 'esmvaltool', 'run', '--config-dir=/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/config', '/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/recipe.yml'] 2025-06-13 13:49:01.850 | ERROR | climate_ref_core.providers:run:417 - 2025-06-13 11:48:58,270 UTC [3518711] INFO ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ Earth System Model Evaluation Tool A community tool for the evaluation of Earth system models. https://esmvaltool.org The Earth System Model Evaluation Tool (ESMValTool) is a community diagnostics and performance metrics tool for the evaluation of Earth System Models (ESMs) that allows for routine comparison of single or multiple models, either against predecessor versions or against observations. Tutorial: https://tutorial.esmvaltool.org Documentation: https://docs.esmvaltool.org Contact: esmvaltool-dev@listserv.dfn.de If you find this software useful for your research, please cite it using https://doi.org/10.5281/zenodo.3387139 for ESMValCore or https://doi.org/10.5281/zenodo.3401363 for ESMValTool or any of the reference papers listed at https://esmvaltool.org/references/. Have fun! 2025-06-13 11:48:58,271 UTC [3518711] INFO Package versions 2025-06-13 11:48:58,271 UTC [3518711] INFO ---------------- 2025-06-13 11:48:58,271 UTC [3518711] INFO ESMValCore: 2.12.0 2025-06-13 11:48:58,271 UTC [3518711] INFO ESMValTool: 2.13.0.dev65+g58fd0b8ec 2025-06-13 11:48:58,271 UTC [3518711] INFO ---------------- 2025-06-13 11:48:58,271 UTC [3518711] INFO Reading configuration files from: /nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/config/configurations/defaults (defaults) /nird/home/yanchun/.config/esmvaltool (default user configuration directory) /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/config (command line argument) 2025-06-13 11:48:58,271 UTC [3518711] INFO Writing program log files to: /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/executions/recipe_20250613_114858/run/main_log.txt /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/executions/recipe_20250613_114858/run/main_log_debug.txt /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/executions/recipe_20250613_114858/run/cmor_log.txt 2025-06-13 11:48:58,505 UTC [3518711] WARNING /nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmpy/interface/loadESMF.py:94: VersionWarning: ESMF installation version 8.8.0, ESMPy version 8.8.0b0 warnings.warn("ESMF installation version {}, ESMPy version {}".format( 2025-06-13 11:48:58,573 UTC [3518711] INFO Starting the Earth System Model Evaluation Tool at time: 2025-06-13 11:48:58 UTC 2025-06-13 11:48:58,573 UTC [3518711] INFO ---------------------------------------------------------------------- 2025-06-13 11:48:58,573 UTC [3518711] INFO RECIPE = /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/recipe.yml 2025-06-13 11:48:58,573 UTC [3518711] INFO RUNDIR = /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/executions/recipe_20250613_114858/run 2025-06-13 11:48:58,575 UTC [3518711] INFO WORKDIR = /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/executions/recipe_20250613_114858/work 2025-06-13 11:48:58,575 UTC [3518711] INFO PREPROCDIR = /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/executions/recipe_20250613_114858/preproc 2025-06-13 11:48:58,575 UTC [3518711] INFO PLOTDIR = /nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/executions/recipe_20250613_114858/plots 2025-06-13 11:48:58,576 UTC [3518711] INFO ---------------------------------------------------------------------- 2025-06-13 11:48:58,576 UTC [3518711] INFO Running tasks using at most 255 processes 2025-06-13 11:48:58,577 UTC [3518711] INFO If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2025-06-13 11:48:58,577 UTC [3518711] INFO If you experience memory problems, try reducing 'max_parallel_tasks' in your configuration. 2025-06-13 11:48:58,598 UTC [3518711] INFO For Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, supplementaries: areacello, *, *, *, *: ignoring supplementary variable 'areacello', unable to expand wildcards 'mip', 'activity', 'ensemble', 'institute', 'exp'. 2025-06-13 11:48:58,599 UTC [3518711] INFO For Dataset: tos, Omon, OBS6, TROPFLUX, v1, supplementaries: areacello, *, *: ignoring supplementary variable 'areacello', unable to expand wildcards 'mip', 'version'. 2025-06-13 11:48:58,599 UTC [3518711] INFO For Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, supplementaries: areacello, *, *, *, *: ignoring supplementary variable 'areacello', unable to expand wildcards 'mip', 'activity', 'ensemble', 'institute', 'exp'. 2025-06-13 11:48:58,600 UTC [3518711] INFO For Dataset: tos, Omon, OBS6, TROPFLUX, v1, supplementaries: areacello, *, *: ignoring supplementary variable 'areacello', unable to expand wildcards 'mip', 'version'. 2025-06-13 11:48:58,601 UTC [3518711] INFO For Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, supplementaries: areacello, *, *, *, *: ignoring supplementary variable 'areacello', unable to expand wildcards 'mip', 'activity', 'ensemble', 'institute', 'exp'. 2025-06-13 11:48:58,601 UTC [3518711] INFO For Dataset: tos, Omon, OBS6, TROPFLUX, v1, supplementaries: areacello, *, *: ignoring supplementary variable 'areacello', unable to expand wildcards 'mip', 'version'. 2025-06-13 11:48:58,602 UTC [3518711] INFO For Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, supplementaries: areacello, *, *, *, *: ignoring supplementary variable 'areacello', unable to expand wildcards 'mip', 'activity', 'ensemble', 'institute', 'exp'. 2025-06-13 11:48:58,602 UTC [3518711] INFO For Dataset: tos, Omon, OBS6, TROPFLUX, v1, supplementaries: areacello, *, *: ignoring supplementary variable 'areacello', unable to expand wildcards 'mip', 'version'. 2025-06-13 11:48:58,604 UTC [3518711] INFO Creating tasks from recipe 2025-06-13 11:48:58,604 UTC [3518711] INFO Creating tasks for diagnostic diagnostic_metrics 2025-06-13 11:48:58,604 UTC [3518711] INFO Creating diagnostic task diagnostic_metrics/plot_script 2025-06-13 11:48:58,604 UTC [3518711] INFO Creating preprocessor task diagnostic_metrics/tos_seas_asym 2025-06-13 11:48:58,604 UTC [3518711] INFO Creating preprocessor 'enso_seas_asym' task for variable 'tos_seas_asym' 2025-06-13 11:48:58,608 UTC [3518711] WARNING Preprocessor function area_statistics works best when at least one supplementary variable of ['areacella', 'areacello'] is defined in the recipe for Dataset: {'diagnostic': 'diagnostic_metrics', 'variable_group': 'tos_seas_asym', 'dataset': 'NorESM2-LM', 'project': 'CMIP6', 'mip': 'Omon', 'short_name': 'tos', 'activity': 'CMIP', 'alias': 'CMIP6', 'ensemble': 'r1i1p1f1', 'exp': 'historical', 'frequency': 'mon', 'grid': 'gn', 'institute': 'NCC', 'long_name': 'Sea Surface Temperature', 'modeling_realm': ['ocean'], 'original_short_name': 'tos', 'preprocessor': 'enso_seas_asym', 'recipe_dataset_index': 0, 'standard_name': 'sea_surface_temperature', 'timerange': '18500116T120000/20141216T120000', 'units': 'degC'} session: 'recipe_20250613_114858'. 2025-06-13 11:48:58,609 UTC [3518711] INFO Found input files for Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, v20190815 2025-06-13 11:48:58,611 UTC [3518711] WARNING Preprocessor function area_statistics works best when at least one supplementary variable of ['areacella', 'areacello'] is defined in the recipe for Dataset: {'diagnostic': 'diagnostic_metrics', 'variable_group': 'tos_seas_asym', 'dataset': 'TROPFLUX', 'project': 'OBS6', 'mip': 'Omon', 'short_name': 'tos', 'alias': 'OBS6', 'frequency': 'mon', 'long_name': 'Sea Surface Temperature', 'modeling_realm': ['ocean'], 'original_short_name': 'tos', 'preprocessor': 'enso_seas_asym', 'recipe_dataset_index': 1, 'standard_name': 'sea_surface_temperature', 'tier': 2, 'type': 'reanaly', 'units': 'degC', 'version': 'v1'} session: 'recipe_20250613_114858'. 2025-06-13 11:48:58,611 UTC [3518711] INFO Found input files for Dataset: tos, Omon, OBS6, TROPFLUX, v1 2025-06-13 11:48:58,612 UTC [3518711] INFO PreprocessingTask diagnostic_metrics/tos_seas_asym created. 2025-06-13 11:48:58,612 UTC [3518711] INFO Creating preprocessor task diagnostic_metrics/tos_amp 2025-06-13 11:48:58,612 UTC [3518711] INFO Creating preprocessor 'enso_amplitude' task for variable 'tos_amp' 2025-06-13 11:48:58,616 UTC [3518711] WARNING /nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/numpy/ma/core.py:5528: RuntimeWarning: Degrees of freedom <= 0 for slice ret = super().var(axis=axis, dtype=dtype, out=out, ddof=ddof, 2025-06-13 11:48:58,617 UTC [3518711] WARNING /nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/numpy/_core/_methods.py:216: RuntimeWarning: invalid value encountered in divide ret = um.true_divide( 2025-06-13 11:48:58,618 UTC [3518711] WARNING Preprocessor function area_statistics works best when at least one supplementary variable of ['areacella', 'areacello'] is defined in the recipe for Dataset: {'diagnostic': 'diagnostic_metrics', 'variable_group': 'tos_amp', 'dataset': 'NorESM2-LM', 'project': 'CMIP6', 'mip': 'Omon', 'short_name': 'tos', 'activity': 'CMIP', 'alias': 'CMIP6', 'ensemble': 'r1i1p1f1', 'exp': 'historical', 'frequency': 'mon', 'grid': 'gn', 'institute': 'NCC', 'long_name': 'Sea Surface Temperature', 'modeling_realm': ['ocean'], 'original_short_name': 'tos', 'preprocessor': 'enso_amplitude', 'recipe_dataset_index': 0, 'standard_name': 'sea_surface_temperature', 'timerange': '18500116T120000/20141216T120000', 'units': 'degC'} session: 'recipe_20250613_114858'. 2025-06-13 11:48:58,619 UTC [3518711] INFO Found input files for Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, v20190815 2025-06-13 11:48:58,622 UTC [3518711] WARNING /nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/numpy/ma/core.py:5528: RuntimeWarning: Degrees of freedom <= 0 for slice ret = super().var(axis=axis, dtype=dtype, out=out, ddof=ddof, 2025-06-13 11:48:58,622 UTC [3518711] WARNING /nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/numpy/_core/_methods.py:216: RuntimeWarning: invalid value encountered in divide ret = um.true_divide( 2025-06-13 11:48:58,623 UTC [3518711] WARNING Preprocessor function area_statistics works best when at least one supplementary variable of ['areacella', 'areacello'] is defined in the recipe for Dataset: {'diagnostic': 'diagnostic_metrics', 'variable_group': 'tos_amp', 'dataset': 'TROPFLUX', 'project': 'OBS6', 'mip': 'Omon', 'short_name': 'tos', 'alias': 'OBS6', 'frequency': 'mon', 'long_name': 'Sea Surface Temperature', 'modeling_realm': ['ocean'], 'original_short_name': 'tos', 'preprocessor': 'enso_amplitude', 'recipe_dataset_index': 1, 'standard_name': 'sea_surface_temperature', 'tier': 2, 'type': 'reanaly', 'units': 'degC', 'version': 'v1'} session: 'recipe_20250613_114858'. 2025-06-13 11:48:58,623 UTC [3518711] INFO Found input files for Dataset: tos, Omon, OBS6, TROPFLUX, v1 2025-06-13 11:48:58,624 UTC [3518711] INFO PreprocessingTask diagnostic_metrics/tos_amp created. 2025-06-13 11:48:58,624 UTC [3518711] INFO Creating preprocessor task diagnostic_metrics/tos_patdiv1 2025-06-13 11:48:58,624 UTC [3518711] INFO Creating preprocessor 'ssta_dec_area' task for variable 'tos_patdiv1' 2025-06-13 11:48:58,627 UTC [3518711] WARNING Preprocessor function area_statistics works best when at least one supplementary variable of ['areacella', 'areacello'] is defined in the recipe for Dataset: {'diagnostic': 'diagnostic_metrics', 'variable_group': 'tos_patdiv1', 'dataset': 'NorESM2-LM', 'project': 'CMIP6', 'mip': 'Omon', 'short_name': 'tos', 'activity': 'CMIP', 'alias': 'CMIP6', 'ensemble': 'r1i1p1f1', 'exp': 'historical', 'frequency': 'mon', 'grid': 'gn', 'institute': 'NCC', 'long_name': 'Sea Surface Temperature', 'modeling_realm': ['ocean'], 'original_short_name': 'tos', 'preprocessor': 'ssta_dec_area', 'recipe_dataset_index': 0, 'standard_name': 'sea_surface_temperature', 'timerange': '18500116T120000/20141216T120000', 'units': 'degC'} session: 'recipe_20250613_114858'. 2025-06-13 11:48:58,628 UTC [3518711] INFO Found input files for Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, v20190815 2025-06-13 11:48:58,631 UTC [3518711] WARNING Preprocessor function area_statistics works best when at least one supplementary variable of ['areacella', 'areacello'] is defined in the recipe for Dataset: {'diagnostic': 'diagnostic_metrics', 'variable_group': 'tos_patdiv1', 'dataset': 'TROPFLUX', 'project': 'OBS6', 'mip': 'Omon', 'short_name': 'tos', 'alias': 'OBS6', 'frequency': 'mon', 'long_name': 'Sea Surface Temperature', 'modeling_realm': ['ocean'], 'original_short_name': 'tos', 'preprocessor': 'ssta_dec_area', 'recipe_dataset_index': 1, 'standard_name': 'sea_surface_temperature', 'tier': 2, 'type': 'reanaly', 'units': 'degC', 'version': 'v1'} session: 'recipe_20250613_114858'. 2025-06-13 11:48:58,631 UTC [3518711] INFO Found input files for Dataset: tos, Omon, OBS6, TROPFLUX, v1 2025-06-13 11:48:58,632 UTC [3518711] INFO PreprocessingTask diagnostic_metrics/tos_patdiv1 created. 2025-06-13 11:48:58,632 UTC [3518711] INFO Creating preprocessor task diagnostic_metrics/tos_lifdur1 2025-06-13 11:48:58,632 UTC [3518711] INFO Creating preprocessor 'ssta_ninoarea' task for variable 'tos_lifdur1' 2025-06-13 11:48:58,635 UTC [3518711] WARNING Preprocessor function area_statistics works best when at least one supplementary variable of ['areacella', 'areacello'] is defined in the recipe for Dataset: {'diagnostic': 'diagnostic_metrics', 'variable_group': 'tos_lifdur1', 'dataset': 'NorESM2-LM', 'project': 'CMIP6', 'mip': 'Omon', 'short_name': 'tos', 'activity': 'CMIP', 'alias': 'CMIP6', 'ensemble': 'r1i1p1f1', 'exp': 'historical', 'frequency': 'mon', 'grid': 'gn', 'institute': 'NCC', 'long_name': 'Sea Surface Temperature', 'modeling_realm': ['ocean'], 'original_short_name': 'tos', 'preprocessor': 'ssta_ninoarea', 'recipe_dataset_index': 0, 'standard_name': 'sea_surface_temperature', 'timerange': '18500116T120000/20141216T120000', 'units': 'degC'} session: 'recipe_20250613_114858'. 2025-06-13 11:48:58,637 UTC [3518711] INFO Found input files for Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, v20190815 2025-06-13 11:48:58,639 UTC [3518711] WARNING Preprocessor function area_statistics works best when at least one supplementary variable of ['areacella', 'areacello'] is defined in the recipe for Dataset: {'diagnostic': 'diagnostic_metrics', 'variable_group': 'tos_lifdur1', 'dataset': 'TROPFLUX', 'project': 'OBS6', 'mip': 'Omon', 'short_name': 'tos', 'alias': 'OBS6', 'frequency': 'mon', 'long_name': 'Sea Surface Temperature', 'modeling_realm': ['ocean'], 'original_short_name': 'tos', 'preprocessor': 'ssta_ninoarea', 'recipe_dataset_index': 1, 'standard_name': 'sea_surface_temperature', 'tier': 2, 'type': 'reanaly', 'units': 'degC', 'version': 'v1'} session: 'recipe_20250613_114858'. 2025-06-13 11:48:58,640 UTC [3518711] INFO Found input files for Dataset: tos, Omon, OBS6, TROPFLUX, v1 2025-06-13 11:48:58,641 UTC [3518711] INFO PreprocessingTask diagnostic_metrics/tos_lifdur1 created. 2025-06-13 11:48:58,641 UTC [3518711] INFO Creating preprocessor task diagnostic_metrics/tos_lifdurdiv2 2025-06-13 11:48:58,641 UTC [3518711] INFO Creating preprocessor 'ssta_meridional' task for variable 'tos_lifdurdiv2' 2025-06-13 11:48:58,645 UTC [3518711] INFO Found input files for Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, v20190815 2025-06-13 11:48:58,648 UTC [3518711] INFO Found input files for Dataset: tos, Omon, OBS6, TROPFLUX, v1 2025-06-13 11:48:58,649 UTC [3518711] INFO PreprocessingTask diagnostic_metrics/tos_lifdurdiv2 created. 2025-06-13 11:48:58,649 UTC [3518711] INFO Creating preprocessor task diagnostic_metrics/tos_pat2 2025-06-13 11:48:58,649 UTC [3518711] INFO Creating preprocessor 'ssta_pattern2' task for variable 'tos_pat2' 2025-06-13 11:48:58,654 UTC [3518711] INFO Found input files for Dataset: tos, Omon, CMIP6, NorESM2-LM, CMIP, historical, r1i1p1f1, gn, v20190815 2025-06-13 11:48:58,657 UTC [3518711] INFO Found input files for Dataset: tos, Omon, OBS6, TROPFLUX, v1 2025-06-13 11:48:58,658 UTC [3518711] INFO PreprocessingTask diagnostic_metrics/tos_pat2 created. 2025-06-13 11:48:58,659 UTC [3518711] INFO These tasks will be executed: diagnostic_metrics/tos_lifdurdiv2, diagnostic_metrics/tos_pat2, diagnostic_metrics/tos_amp, diagnostic_metrics/tos_patdiv1, diagnostic_metrics/plot_script, diagnostic_metrics/tos_lifdur1, diagnostic_metrics/tos_seas_asym 2025-06-13 11:48:58,673 UTC [3518711] INFO Wrote recipe with version numbers and wildcards to: file:///nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/executions/recipe_20250613_114858/run/recipe_filled.yml 2025-06-13 11:48:58,673 UTC [3518711] INFO Using Dask threaded scheduler. The distributed scheduler is recommended, please read https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#dask-configuration how to use a distributed scheduler. 2025-06-13 11:48:58,673 UTC [3518711] INFO Running 7 tasks using 7 processes 2025-06-13 11:48:58,673 UTC [3518711] INFO Using the threaded Dask scheduler with 40 worker threads per preprocessing task. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#f5 for more information. 2025-06-13 11:48:58,739 UTC [3518867] INFO Starting task diagnostic_metrics/tos_seas_asym in process [3518867] 2025-06-13 11:48:58,739 UTC [3518868] INFO Starting task diagnostic_metrics/tos_amp in process [3518868] 2025-06-13 11:48:58,739 UTC [3518869] INFO Starting task diagnostic_metrics/tos_patdiv1 in process [3518869] 2025-06-13 11:48:58,740 UTC [3518870] INFO Starting task diagnostic_metrics/tos_lifdur1 in process [3518870] 2025-06-13 11:48:58,740 UTC [3518871] INFO Starting task diagnostic_metrics/tos_lifdurdiv2 in process [3518871] 2025-06-13 11:48:58,741 UTC [3518872] INFO Starting task diagnostic_metrics/tos_pat2 in process [3518872] 2025-06-13 11:48:58,836 UTC [3518711] INFO Progress: 6 tasks running, 1 tasks waiting for ancestors, 0/7 done 2025-06-13 11:49:01,000 UTC [3518867] ERROR Supplementary variables are needed to calculate grid cell areas for irregular or unstructured grid of cube sea_surface_temperature / (degC) (time: 1980; cell index along second dimension: 39; cell index along first dimension: 50) 2025-06-13 11:49:01,002 UTC [3518867] ERROR Failed to run preprocessor function 'area_statistics' on the data [] loaded from original input file(s) [LocalFile('/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/climate_data/CMIP6/CMIP/NCC/NorESM2-LM/historical/r1i1p1f1/Omon/tos/gn/v20190815/tos_Omon_NorESM2-LM_historical_r1i1p1f1_gn_185001-185912.nc'), LocalFile('/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/climate_data/CMIP6/CMIP/NCC/NorESM2-LM/historical/r1i1p1f1/Omon/tos/gn/v20190815/tos_Omon_NorESM2-LM_historical_r1i1p1f1_gn_186001-186912.nc'), LocalFile('/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/climate_data/CMIP6/CMIP/NCC/NorESM2-LM/historical/r1i1p1f1/Omon/tos/gn/v20190815/tos_Omon_NorESM2-LM_historical_r1i1p1f1_gn_187001-187912.nc'), LocalFile('/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/climate_data/CMIP6/CMIP/NCC/NorESM2-LM/historical/r1i1p1f1/Omon/tos/gn/v20190815/tos_Omon_NorESM2-LM_historical_r1i1p1f1_gn_188001-188912.nc')] (and 13 further file(s) not shown here; refer to the debug log for a full list) with function argument(s) operator = 'mean' 2025-06-13 11:49:01,298 UTC [3518711] INFO Maximum memory used (estimate): 4.9 GB 2025-06-13 11:49:01,300 UTC [3518711] INFO Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2025-06-13 11:49:01,300 UTC [3518711] ERROR Program terminated abnormally, see stack trace below for more information: multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) ^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_task.py", line 988, in _run_task output_files = task.run() ^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_task.py", line 293, in run self.output_files = self._run(input_files) ^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 727, in _run product.apply(step, self.debug) File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 525, in apply self.cubes = preprocess( ^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 429, in preprocess _run_preproc_function( File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/preprocessor/__init__.py", line 367, in _run_preproc_function return function(items, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/preprocessor/_shared.py", line 269, in wrapper result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/preprocessor/_area.py", line 356, in area_statistics agg_kwargs = update_weights_kwargs( ^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/preprocessor/_shared.py", line 174, in update_weights_kwargs callback(cube, **callback_kwargs) File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/preprocessor/_shared.py", line 500, in try_adding_calculated_cell_area raise CoordinateMultiDimError(cube.coord("latitude")) iris.exceptions.CoordinateMultiDimError: Multi-dimensional coordinate not supported: 'latitude' """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_main.py", line 603, in run fire.Fire(ESMValTool()) File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( ^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_main.py", line 429, in run self._run(recipe, session, cli_config_dir) File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_main.py", line 486, in _run process_recipe(recipe_file=recipe, session=session) File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_main.py", line 135, in process_recipe recipe.run() File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_recipe/recipe.py", line 1200, in run self.tasks.run(max_parallel_tasks=self.session["max_parallel_tasks"]) File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_task.py", line 839, in run self._run_parallel(address, max_parallel_tasks) File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_task.py", line 946, in _run_parallel _copy_results(task, running[task]) File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/site-packages/esmvalcore/_task.py", line 976, in _copy_results task.output_files, task.products = future.get() ^^^^^^^^^^^^ File "/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d/lib/python3.12/multiprocessing/pool.py", line 774, in get raise self._value iris.exceptions.CoordinateMultiDimError: Multi-dimensional coordinate not supported: 'latitude' 2025-06-13 11:49:01,303 UTC [3518711] INFO If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory. 2025-06-13 13:49:01.853 | ERROR | climate_ref_core.logging:redirect_logs:199 - Execution failed Traceback (most recent call last): File "", line 1, in File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/multiprocessing/spawn.py", line 122, in spawn_main exitcode = _main(fd, parent_sentinel) │ │ └ 3 │ └ 16 └ File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/multiprocessing/spawn.py", line 135, in _main return self._bootstrap(parent_sentinel) │ │ └ 3 │ └ └ File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap self.run() │ └ └ File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) │ │ │ │ │ └ {} │ │ │ │ └ │ │ │ └ (, │ └ └ File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/concurrent/futures/process.py", line 261, in _process_worker r = call_item.fn(*call_item.args, **call_item.kwargs) │ │ │ │ │ └ {'definition': ExecutionDefinition(diagnostic=ENSOCharacteristics(name='ENSO Characteristics'), key='cmip6_gn_r1i1p1f1_NorESM... │ │ │ │ └ │ │ │ └ () │ │ └ │ └ └ File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/site-packages/climate_ref/executor/local.py", line 87, in _process_run return execute_locally(definition=definition, log_level=log_level) │ │ └ 'INFO' │ └ ExecutionDefinition(diagnostic=ENSOCharacteristics(name='ENSO Characteristics'), key='cmip6_gn_r1i1p1f1_NorESM2-LM', datasets... └ File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/site-packages/climate_ref_core/executor.py", line 48, in execute_locally with redirect_logs(definition, log_level): │ │ └ 'INFO' │ └ ExecutionDefinition(diagnostic=ENSOCharacteristics(name='ENSO Characteristics'), key='cmip6_gn_r1i1p1f1_NorESM2-LM', datasets... └ File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) │ │ │ │ │ └ │ │ │ │ └ CalledProcessError(1, ['/nird/datalake/NS16000B/climate-ref-software/conda/micromamba', 'run', '--prefix', '/nird/datalake/NS... │ │ │ └ │ │ └ │ └ └ > File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/site-packages/climate_ref_core/logging.py", line 197, in redirect_logs yield File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/site-packages/climate_ref_core/executor.py", line 49, in execute_locally return definition.diagnostic.run(definition=definition) │ │ └ ExecutionDefinition(diagnostic=ENSOCharacteristics(name='ENSO Characteristics'), key='cmip6_gn_r1i1p1f1_NorESM2-LM', datasets... │ └ └ ExecutionDefinition(diagnostic=ENSOCharacteristics(name='ENSO Characteristics'), key='cmip6_gn_r1i1p1f1_NorESM2-LM', datasets... File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/site-packages/climate_ref_core/diagnostics.py", line 538, in run self.execute(definition) │ │ └ ExecutionDefinition(diagnostic=ENSOCharacteristics(name='ENSO Characteristics'), key='cmip6_gn_r1i1p1f1_NorESM2-LM', datasets... │ └ └ ENSOCharacteristics(name='ENSO Characteristics') File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/site-packages/climate_ref_core/diagnostics.py", line 582, in execute self.provider.run(cmd) │ │ └ ['esmvaltool', 'run', '--config-dir=/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4... │ └ └ ENSOCharacteristics(name='ENSO Characteristics') File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/site-packages/climate_ref_core/providers.py", line 418, in run raise e File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/site-packages/climate_ref_core/providers.py", line 406, in run res = subprocess.run( # noqa: S603 │ └ └ File "/nird/datalake/NS16000B/climate-ref-env/lib/python3.11/subprocess.py", line 571, in run raise CalledProcessError(retcode, process.args, │ │ │ └ ['/nird/datalake/NS16000B/climate-ref-software/conda/micromamba', 'run', '--prefix', '/nird/datalake/NS16000B/climate-ref-sof... │ │ └ │ └ 1 └ subprocess.CalledProcessError: Command '['/nird/datalake/NS16000B/climate-ref-software/conda/micromamba', 'run', '--prefix', '/nird/datalake/NS16000B/climate-ref-software/conda/esmvaltool-9776cff81b525ef5fc839c1b70ccfa40943cee1d', 'esmvaltool', 'run', '--config-dir=/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/config', '/nird/scratch/yanchun/diagnostics/climate-ref/scratch/esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09/recipe.yml']' returned non-zero exit status 1. 2025-06-13 13:49:01.862 | INFO | climate_ref_core.logging:redirect_logs:202 - Diagnostic execution complete. Results available in esmvaltool/enso-characteristics/9b4e64398137132d913ecb4a73985377e4e79f09