Running tests

This guide will break down how to run calculations, analysis, and the interactive application.

Calculations

All calculations can be launched using our ml_peg calc command-line command.

Help for this command can be found by running ml_peg calc --help:

Usage: ml_peg calc [OPTIONS]

Run calculations

╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --models                                 TEXT  Comma-separated models to run calculations on. Default is all models.      │
│ --category                               TEXT  Category to run calculations for. Default is all categories. [default: *]  │
│ --test                                   TEXT  Test to run calculations for. Default is all tests. [default: *]           │
│ --run-slow         --no-run-slow               Whether to run calculations labelled slow. [default: run-slow]             │
│ --run-very-slow    --no-run-very-slow          Whether to run calculations labelled very slow.                            │
│                                                [default: no-run-very-slow]                                                │
│ --verbose          --no-verbose                Whether to run pytest with verbose and stdout printed. [default: verbose]  │
│ --help                                         Show this message and exit.                                                │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

ml_peg calc launches calculations using pytest, and will automatically discover and run each test, handle intermediate errors, and control which tests are run based on our custom markers.

For example, to run the S24 test in the surfaces category, with the mace-mp-0b3 model, you could run:

ml_peg calc --category surfaces --test S24 --models mace-mp-0b3

This is effectively equivalent to:

.. code-block:: bash

pytest -vvv ml_peg/calcs/surfaces/S24/calc_S24.py --models mace-mp-0b3

Analysis

Similarly to calculations, analysis can be launched using our ml_peg analyse command-line command.

Help for this command can be found by running ml_peg analyse --help:

Usage: ml_peg analyse [OPTIONS]

Run calculations

╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --models                      TEXT  Comma-separated models to run analysis for. Default is all models.         │
│ --category                    TEXT  Category to run analysis for. Default is all categories. [default: *]      │
│ --test                        TEXT  Test to run analysis for. Default is all tests. [default: *]               │
│ --verbose     --no-verbose          Whether to run pytest with verbose and stdout printed. [default: verbose]  │
│ --help                              Show this message and exit.                                                │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

ml_peg analyse launches analysis using pytest.

For example, to run the OC157 test in the surfaces category, with the mace-mp-0b3 and orb-v3-consv-inf-omat models, you could run:

ml_peg analyse --category surfaces --test OC157 --models mace-mp-0b3,orb-v3-consv-inf-omat

This is effectively equivalent to:

.. code-block:: bash

pytest -vvv ml_peg/analysis/surfaces/OC157/analyse_OC157.py --models mace-mp-0b3,orb-v3-consv-inf-omat

Application

Having run analysis, the app can now be launched by running the ml_peg app command-line command.

Help for this command can be found by running ml_peg app --help:

Usage: ml_peg app [OPTIONS]

Run application

╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --models                    TEXT  Comma-separated models to build interactivity for. Default is all models. │
│ --category                  TEXT  Category to build app for. Default is all categories. [default: *]        │
│ --port                      TEXT  Port to run application on. [default: 8050]                               │
│ --debug       --no-debug          Whether to run with Dash debugging. [default: debug]                      │
│ --help                            Show this message and exit.                                               │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Note

The models option for this command only influences building interactive callbacks, and does not change whether the models are included in tables, scores, or summaries,

When launched, the app will attempt to automatically construct tables, figures, and interactive features, based on any importable test apps defined in ml_peg/apps/.

If any plots are unable to be loaded, a warning will be raised, and only the table will be rendered for the test.

If a test’s table is also unable to be loaded, the test will not be added to the app, but the app builder should continue to attempt adding other tests.

By default, the live app can then be accessed at http://localhost:8050.

To run the app on a different port (e.g. 8060), and for only the NEBs category, run:

ml_peg app --category nebs --port 8060