INFO [1956658] ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ Earth System Model Evaluation Tool A community tool for the evaluation of Earth system models. https://esmvaltool.org The Earth System Model Evaluation Tool (ESMValTool) is a community diagnostics and performance metrics tool for the evaluation of Earth System Models (ESMs) that allows for routine comparison of single or multiple models, either against predecessor versions or against observations. Tutorial: https://tutorial.esmvaltool.org Documentation: https://docs.esmvaltool.org Contact: esmvaltool-dev@listserv.dfn.de If you find this software useful for your research, please cite it using https://doi.org/10.5281/zenodo.3387139 for ESMValCore or https://doi.org/10.5281/zenodo.3401363 for ESMValTool or any of the reference papers listed at https://esmvaltool.org/references/. Have fun! INFO [1956658] Package versions INFO [1956658] ---------------- INFO [1956658] ESMValCore: 2.15.0 INFO [1956658] ESMValTool: 2.15.0 INFO [1956658] ---------------- INFO [1956658] Reading configuration files from: /nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/config/configurations/defaults (defaults) /nird/home/yanchun/.config/esmvaltool (default user configuration directory) INFO [1956658] Writing program log files to: /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/run/main_log.txt /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/run/main_log_debug.txt /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/run/cmor_log.txt INFO [1956658] Starting the Earth System Model Evaluation Tool at time: 2026-10-26 12:21:53 UTC INFO [1956658] ---------------------------------------------------------------------- INFO [1956658] RECIPE = /nird/home/yanchun/diagnostics/esmvaltool/esmvaltool/recipes/examples/recipe_python.yml INFO [1956658] RUNDIR = /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/run INFO [1956658] WORKDIR = /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/work INFO [1956658] PREPROCDIR = /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/preproc INFO [1956658] PLOTDIR = /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/plots INFO [1956658] ---------------------------------------------------------------------- INFO [1956658] Running tasks using at most 255 processes INFO [1956658] If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. INFO [1956658] If you experience memory problems, try reducing 'max_parallel_tasks' in your configuration. INFO [1956658] Creating tasks from recipe INFO [1956658] Creating tasks for diagnostic map INFO [1956658] Creating diagnostic task map/script1 INFO [1956658] Creating preprocessor task map/tas INFO [1956658] Creating preprocessor 'to_degrees_c' task for variable 'tas' INFO [1956658] Found input files for Dataset: tas, Amon, CMIP6, CESM2, historical, r1i1p1f1, mon, gn, v20190308 INFO [1956658] Found input files for Dataset: tas, Amon, CMIP6, NorESM2-LM, historical, r1i1p1f1, mon, gn, v20190815 INFO [1956658] PreprocessingTask map/tas created. INFO [1956658] Creating tasks for diagnostic timeseries INFO [1956658] Creating diagnostic task timeseries/script1 INFO [1956658] Creating preprocessor task timeseries/tas_amsterdam INFO [1956658] Creating preprocessor 'annual_mean_amsterdam' task for variable 'tas_amsterdam' INFO [1956658] Found input files for Dataset: tas, Amon, CMIP6, CESM2, historical, r1i1p1f1, mon, gn, v20190308 INFO [1956658] Found input files for Dataset: tas, Amon, CMIP6, NorESM2-LM, historical, r1i1p1f1, mon, gn, v20190815 INFO [1956658] PreprocessingTask timeseries/tas_amsterdam created. INFO [1956658] Creating preprocessor task timeseries/tas_global INFO [1956658] Creating preprocessor 'annual_mean_global' task for variable 'tas_global' INFO [1956658] Found input files for Dataset: tas, Amon, CMIP6, CESM2, historical, r1i1p1f1, mon, gn, v20190308, supplementaries: areacella, fx, fx INFO [1956658] Found input files for Dataset: tas, Amon, CMIP6, NorESM2-LM, historical, r1i1p1f1, mon, gn, v20190815, supplementaries: areacella, fx, fx INFO [1956658] PreprocessingTask timeseries/tas_global created. INFO [1956658] These tasks will be executed: map/script1, timeseries/tas_global, timeseries/script1, map/tas, timeseries/tas_amsterdam INFO [1956658] Wrote recipe with version numbers and wildcards to: file:///nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/run/recipe_python_filled.yml INFO [1956658] Downloading missing data (this may take a while...). Details can be found in the debug log at /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/run/main_log_debug.txt INFO [1956658] Successfully downloaded missing data INFO [1956658] Using Dask threaded scheduler. The distributed scheduler is recommended, please read https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#dask-configuration how to use a distributed scheduler. INFO [1956658] Running 5 tasks using 5 processes INFO [1956658] Using the threaded Dask scheduler with 79 worker threads per preprocessing task. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#f5 for more information. INFO [1957939] Starting task map/tas in process [1957939] INFO [1957940] Starting task timeseries/tas_amsterdam in process [1957940] INFO [1957941] Starting task timeseries/tas_global in process [1957941] INFO [1956658] Progress: 3 tasks running, 2 tasks waiting for ancestors, 0/5 done INFO [1957940] Extracting data for Amsterdam, Noord-Holland, Nederland (52.3730796 °N, 4.8924534 °E) INFO [1957939] Computing and saving data for preprocessing task map/tas INFO [1957940] Extracting data for Amsterdam, Noord-Holland, Nederland (52.3730796 °N, 4.8924534 °E) ERROR [1957940] Failed to run preprocessor function 'multi_model_statistics' on the data [PreprocessorFile: /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/preproc/timeseries/tas_amsterdam/CMIP6_NorESM2-LM_Amon_historical_r1i1p1f1_tas_gn_1850-2000.nc, PreprocessorFile: /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/preproc/timeseries/tas_amsterdam/CMIP6_CESM2_Amon_historical_r1i1p1f1_tas_gn_1850-2000.nc] with function argument(s) statistics = ['mean'], span = 'overlap', output_products = defaultdict(, {'': {'mean': PreprocessorFile: /nird/datalake/NS2345K/www/diagnostics/esmvaltool/yanchun/tmp/recipe_python_20261026_122152/preproc/timeseries/tas_amsterdam/CMIP6_MultiModelMean_historical_Amon_tas_1850-2000.nc}}) INFO [1956658] Maximum memory used (estimate): 7.1 GB INFO [1956658] Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. ERROR [1956658] Program terminated abnormally, see stack trace below for more information: multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/_multimodel.py", line 447, in _combine merged_cube = cubes.merge_cube() File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/iris/cube.py", line 364, in merge_cube proto_cube.register(c, error_on_mismatch=True) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/iris/_merge.py", line 1334, in register match = coord_payload.match_signature( self._coord_signature, error_on_mismatch ) File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/iris/_merge.py", line 272, in match_signature raise iris.exceptions.MergeError(msgs) iris.exceptions.MergeError: failed to merge into a single cube. Coordinates in cube.aux_coords (scalar) differ by dtype or class (i.e. DimCoord vs AuxCoord). The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) ~~~~^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_task.py", line 1015, in _run_task output_files = task.run(session) File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_task.py", line 295, in run self.output_files = self._run(input_files, session=session) ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/__init__.py", line 879, in _run self.products = _apply_multimodel( ~~~~~~~~~~~~~~~~~^ self.products, ^^^^^^^^^^^^^^ ...<2 lines>... self.debug, ^^^^^^^^^^^ ) ^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/__init__.py", line 824, in _apply_multimodel result: list[PreprocessorFile] = preprocess( # type: ignore ~~~~~~~~~~^^^^^^^^^^^^^^^^^ products - exclude, # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ step, ^^^^^ **settings, ^^^^^^^^^^^ ) ^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/__init__.py", line 571, in preprocess _run_preproc_function( ~~~~~~~~~~~~~~~~~~~~~^ function, ^^^^^^^^^ ...<2 lines>... input_files=input_files, ^^^^^^^^^^^^^^^^^^^^^^^^ ), ^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/__init__.py", line 511, in _run_preproc_function return function(items, **kwargs) File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/_multimodel.py", line 821, in multi_model_statistics group_statistics = _multiproduct_statistics( products=input_prods, ...<4 lines>... ignore_scalar_coords=ignore_scalar_coords, ) File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/_multimodel.py", line 644, in _multiproduct_statistics statistics_cubes = _multicube_statistics( cubes=cubes, ...<2 lines>... ignore_scalar_coords=ignore_scalar_coords, ) File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/_multimodel.py", line 621, in _multicube_statistics combined_cube = _combine(cubes) File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/_multimodel.py", line 456, in _combine raise ValueError(msg) from exc ValueError: Multi-model statistics failed to merge input cubes into a single array: 0: air_temperature / (degrees_C) (time: 151) 1: air_temperature / (degrees_C) (time: 151) Coordinates in cube.aux_coords (scalar) differ by dtype or class (i.e. DimCoord vs AuxCoord). """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_main.py", line 828, in run fire.Fire(ESMValTool()) ~~~~~~~~~^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( ~~~~~~~~~~~~~~~~~~~^ component, ^^^^^^^^^^ ...<2 lines>... treatment='class' if is_class else 'routine', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ target=component.__name__) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_main.py", line 664, in run self._run(recipe, session, cli_config_dir) ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_main.py", line 717, in _run process_recipe(recipe_file=recipe, session=session) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_main.py", line 154, in process_recipe recipe.run() ~~~~~~~~~~^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_recipe/recipe.py", line 1343, in run self.tasks.run(self.session) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_task.py", line 848, in run self._run_parallel(address, session) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_task.py", line 968, in _run_parallel _copy_results(task, running[task]) ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_task.py", line 998, in _copy_results task.output_files, task.products = future.get() ~~~~~~~~~~^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/multiprocessing/pool.py", line 774, in get raise self._value File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) ^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_task.py", line 1015, in _run_task output_files = task.run(session) ^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/_task.py", line 295, in run self.output_files = self._run(input_files, session=session) ^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/__init__.py", line 879, in _run self.products = _apply_multimodel( ^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/__init__.py", line 824, in _apply_multimodel result: list[PreprocessorFile] = preprocess( # type: ignore ^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/__init__.py", line 571, in preprocess _run_preproc_function( ^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/__init__.py", line 511, in _run_preproc_function return function(items, **kwargs) ^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/_multimodel.py", line 821, in multi_model_statistics group_statistics = _multiproduct_statistics( ^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/_multimodel.py", line 644, in _multiproduct_statistics statistics_cubes = _multicube_statistics( ^^^^^^^^^^^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/_multimodel.py", line 621, in _multicube_statistics combined_cube = _combine(cubes) ^^^^^^^ File "/nird/datalake/NS16000B/esmvaltool-env/2.15.0/lib/python3.13/site-packages/esmvalcore/preprocessor/_multimodel.py", line 456, in _combine raise ValueError(msg) from exc ^^^^^^^^^^^^^^^ ValueError: Multi-model statistics failed to merge input cubes into a single array: 0: air_temperature / (degrees_C) (time: 151) 1: air_temperature / (degrees_C) (time: 151) Coordinates in cube.aux_coords (scalar) differ by dtype or class (i.e. DimCoord vs AuxCoord). INFO [1956658] If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory.