API Reference

Contents

API Reference#

Quick reference#

Function

Description

compute_scaledRI

Compute Scaled RI (RI₀₋₃ by default) from six isoGDGT abundances

predict_proxy_from_T

Forward: temperature → proxy percentiles (pure Python)

predict_T_from_proxyObs

Inverse: proxy → temperature with full uncertainty (runs Stan)

compute_quality_flags

Flag reconstructed samples the calibration cannot support

download_posteriors

Download forward posteriors from Zenodo

download_training_data

Download training CSVs, CMEMS NO₃ uncertainty field + ocean_prop_ds

list_posteriors

Print and return posterior names in the local cache and bundled with the package

MahalanobisOutlierDetector

Flag samples outside the published calibration domain (no fitting required)

InvTConfig

Control the number of forward draws M a reconstruction marginalises over

build_fwd_data

Build validated Stan data dict for forward calibration

get_posterior

Run forward calibration Stan sampling

save_posterior

Persist forward posterior as compressed NetCDF

load_posterior

Load a forward or invT posterior from the cache

summarize_sampler_diagnostics

Divergences, R-hat, ESS, E-BFMI


Prediction#

Compute Scaled Ring Index#

compute_scaledRI(gdgt0, gdgt1, gdgt2, gdgt3, cren, cren_prime, *, cren_weight=3, cren_rings=None)[source]#

Compute Scaled Ring Index from six isoGDGT abundances.

Accepts raw LC/MS peak areas or fractional abundances — both give identical results because the formula divides by the total sum of all six GDGTs, so any common scale factor drops out. Default cren_weight=3 produces scaledRI_cren3 (RI₀₋₃), the canonical proxy used in TEXAS calibration posteriors.

Parameters:
  • gdgt0 (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).

  • gdgt1 (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).

  • gdgt2 (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).

  • gdgt3 (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).

  • cren (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).

  • cren_prime (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).

  • cren_weight (float) – Weight carried by both crenarchaeol and its regioisomer, which also sets the normalization (see Notes). 3 → scaledRI_cren3 / RI₀₋₃ (default, recommended). 4 → scaledRI / RI₀₋₄ (Zhang et al. 2016 convention).

  • cren_rings (int, optional) – Deprecated alias for cren_weight. Emits a DeprecationWarning.

Returns:

Scaled Ring Index, dimensionless, nominally in [0, 1].

Return type:

numpy.ndarray or float

Notes

The formula (paper Eqs. 6 and 8 for the default cren_weight=3) is:

RI      = (1·GDGT1 + 2·GDGT2 + 3·GDGT3 + cren_weight·cren + cren_weight·cren')
          / (GDGT0 + GDGT1 + GDGT2 + GDGT3 + cren + cren')
scaledRI = RI / cren_weight

The parameter does two jobs at once: it is the coefficient on cren and cren’ in the numerator and the constant the whole index is divided by, so it fixes the scale on which every sample is expressed, not just the two crenarchaeol terms. That second job is what makes a value chosen here part of the proxy’s definition rather than a detail — a posterior calibrated at 3 cannot read an index built at 4.

The older name cren_rings is still accepted, with a deprecation warning. It was misleading twice over: it named only the first job, and it implied a count of cyclic moieties. Neither 3 nor 4 is that count. They are calibration conventions, which is why both are offered and why neither is “correct”.

Examples

>>> compute_scaledRI(0.45, 0.10, 0.08, 0.05, 0.30, 0.02)
array(0.45666...)
>>> import pandas as pd
>>> df = pd.read_csv("my_gdgt_data.csv")
>>> df["scaledRI_cren3"] = compute_scaledRI(
...     df["GDGT-0"], df["GDGT-1"], df["GDGT-2"], df["GDGT-3"],
...     df["cren"],   df["cren_prime"],
... )

Predict proxy from T#

predict_proxy_from_T(temperatures, posterior, *, n_draws=500, percentiles=[5, 50, 95], return_full=False, seed=42, gdgt23ratio=None, no3=None, no3_cutoff=None, suffix=None, fwd_cache_dir=None)[source]#

Forward prediction: temperature → proxy percentiles (Scaled RI, TEX86, or any fitted proxy).

Samples n_draws self-consistent parameter sets from the forward calibration posterior (all parameters drawn from the same posterior index, preserving correlations) and evaluates the calibration curve at each requested temperature. This is the forward model of the paper: the generalized logistic curve of Eq. 9 and, for multivariate calibrations, the T₀ shift of Eq. 13 (older additive posteriors add their corrections to the curve instead).

Parameters:
  • temperatures (array-like) – Temperatures (°C) at which to evaluate the calibration curve.

  • posterior (xr.Dataset or str) – Forward calibration posterior — either a loaded xr.Dataset or a saved-file name string (looked up in the posterior cache).

  • n_draws (int) – Number of posterior draws to sample. Default 500.

  • percentiles (list of float) – Percentiles to return, e.g. [5, 50, 95].

  • return_full (bool) – If True, also return the full (n_draws × len(temperatures)) ensemble array and run metadata under keys "ensemble" and "metadata".

  • seed (int) – Random seed for reproducible draw sampling.

  • gdgt23ratio (array-like, optional) – GDGT-2/GDGT-3 ratio values (one per temperature point). Required only when the posterior was fitted with a G₂/₃ term (γ_{G₂/₃} in T₀-shift posteriors, β_{G₂/₃} in older additive ones).

  • no3 (array-like, optional) – Nitrate concentration values (one per temperature point). Required only when the posterior was fitted with NO₃ correction.

  • no3_cutoff (float, optional) – Nitrate threshold (μmol/L) below which the NO₃ correction applies. Defaults to the value stored in the posterior attributes.

  • suffix (str, optional) – Force a specific parameter suffix (e.g. "crtp"). Auto-detected by priority order when omitted.

  • fwd_cache_dir (Path or str, optional) – Directory to resolve posterior in when it is given as a name string. Defaults to the standard forward posterior cache. Ignored when a loaded Dataset is passed.

Returns:

"x_vals" — temperature array (°C) "pN" — one key per requested percentile, e.g. "p5", "p50", "p95" "ensemble" — full array, shape (n_draws, len(temperatures)), if return_full=True "metadata" — run metadata dict, if return_full=True

Return type:

dict with keys

Predict T from proxy observations#

predict_T_from_proxyObs(proxyObs, prior_mu_t, prior_sigma_t, fwd_posterior=None, *, proxy_name=None, temptype=None, site_name=None, predictors=None, no3=None, gdgt23ratio=None, site_lat=None, site_lon=None, no3_dataset=None, no3_dataset_var='no3_sf2tc_avg', flags=True, tex86=None, config=None, chains=4, iter_warmup=500, iter_sampling=1000, seed=42, threads_per_chain=None, save_results=False, save_draws=False, filename_tag=None, cache_dir=None, fwd_cache_dir=None)[source]#

Reconstruct temperature from proxy observations, with full uncertainty.

Marginalises over M draws from the forward calibration posterior so calibration uncertainty propagates into the reconstruction. See TEXAS Proxy System Model — Bayesian Inference Workflow Section 8 for the default calibration, the NO3 resolution order, and what the quality flags mean.

Parameters:
  • proxyObs (ndarray | List[float]) – Observed proxy values, shape (N,) – scaledRI, TEX86, …

  • prior_mu_t (ndarray | float) – Prior mean temperature (degC), scalar or shape (N,).

  • prior_sigma_t (float) – Prior temperature SD (degC); use ~10 when little is known.

  • fwd_posterior (str | Dataset | None) – Forward calibration: a case id/legacy name (str, loaded from the cache) or a pre-loaded xr.Dataset. If omitted, a bundled calibration for temptype is used: the multivariate one (G23 + NO3) when any predictor is given, otherwise the temperature-only one (with a warning).

  • proxy_name (str | None) – Proxy label. Inherited from the calibration when omitted.

  • temptype (str | None) – "SST" or "thermoT"; a label, except it picks the default calibration when fwd_posterior is omitted.

  • site_name (str | None) – Label for the metadata and output filenames.

  • predictors (Dict[str, ndarray] | None) – Non-thermal predictor arrays, e.g. {"gdgt23ratio": ..., "no3": ...}.

  • no3 (float | ndarray | None) – Nitrate (umol/L), scalar or shape (N,). Overrides predictors.

  • gdgt23ratio (float | ndarray | None) – GDGT-2/3 ratio, scalar or shape (N,). Overrides predictors.

  • site_lat (float | ndarray | None) – Latitude(s) for a modern NO3 lookup in the CMEMS nitrate climatology (ocean_prop_ds). Needs site_lon.

  • site_lon (float | ndarray | None) – Longitude(s) for the same lookup.

  • no3_dataset (Dataset | None) – The ocean_prop_ds (lat, lon) field (CMEMS nitrate on the WOA23 grid); downloaded from Zenodo and cached if omitted.

  • no3_dataset_var (str) – Variable to read from it. Default "no3_sf2tc_avg".

  • flags (bool) – Attach result["flags"], one row per observation. Default True.

  • tex86 (float | ndarray | None) – TEX86 for the same samples; only outside_domain uses it.

  • config (InvTConfig | None) – InvTConfig controlling M, the seed and the suffix.

  • chains (int) – MCMC chains. Default 4.

  • iter_warmup (int) – Warmup iterations per chain. Default 500.

  • iter_sampling (int) – Sampling iterations per chain. Default 1000.

  • seed (int) – Random seed. Default 42.

  • threads_per_chain (int | None) – Within-chain parallelism for reduce_sum models.

  • save_results (bool) – Write the quantile .nc and results .npz.

  • save_draws (bool) – Also write the raw draws as {base}_draws.nc.

  • filename_tag (str | Sequence[str] | None) – Extra tag(s) for the output filenames.

  • cache_dir (str | Path | None) – Where outputs are written. Defaults to the invT cache.

  • fwd_cache_dir (str | Path | None) – Where a named fwd_posterior is read from; a different directory from cache_dir.

Returns:

A dict with "proxyObs", "proxy_name", "metadata", one "pN" array per posterior quantile ("p5", "p50", …), and "flags" when flags is True.

Return type:

Dict[str, Any]

Per-observation quality flags#

Returned automatically as result["flags"], and callable on its own against a saved reconstruction. What each column means, and how posterior_prior_width_ratio relates to the manuscript’s Resolved diagnostic, is on the PSM page.

compute_quality_flags(proxyObs, fwd_posterior, *, predictors=None, prior_sigma_t=None, result=None, tex86=None, suffix=None, domain_confidence=0.9, prior_dominated_ratio=0.85)[source]#

Flag observations a reconstruction cannot support, one row per input.

Parameters:
  • proxyObs (array-like, shape (N,)) – The observations handed to the inverse model.

  • fwd_posterior (xr.Dataset) – The forward calibration the reconstruction used. Both the parameter draws and the attrs recording the calibration’s own data range are read from it.

  • predictors (dict, optional) – {"gdgt23ratio": ..., "no3": ...} as passed to the reconstruction. Scalars are broadcast. Absent predictors that the calibration uses are reported by predictor_missing.

  • prior_sigma_t (float, optional) – The temperature prior SD. Required for prior_dominated; without it that column is pd.NA.

  • result (dict, optional) – The reconstruction result. prior_dominated needs p16/p84 from it; without it that column is pd.NA.

  • tex86 (float or array-like, optional) – TEX86 for the same samples. The published calibration-domain ellipse is two-dimensional (TEX86 x Scaled RI), so outside_domain can only be evaluated when both are available; it is pd.NA otherwise.

  • suffix (str, optional) – Parameter suffix to read ("crtp", "culmeso", …). Chosen by the usual priority when omitted.

  • domain_confidence (float, default 0.90) – Confidence level of the Mahalanobis ellipse.

  • prior_dominated_ratio (float, default 0.85) – Posterior/prior SD ratio at or above which a row is prior-dominated.

Returns:

N rows. Boolean columns from FLAG_COLUMNS, graded columns from DIAGNOSTIC_COLUMNS, and any_flag.

A check that could not be evaluated is pd.NA, never False — “not assessed” and “passed” are different answers, and collapsing them would quietly launder the first into the second. any_flag therefore ORs only what was actually evaluated, and only the defect columns: ADVISORY_COLUMNS are reported but do not vote.

Return type:

pd.DataFrame


Download and cache#

Download posteriors#

download_posteriors(names=None, cache_dir=None, force=False)[source]#

Download forward calibration posteriors from Zenodo.

Parameters:
  • names (list of str, optional) – Subset of POSTERIOR_REGISTRY keys to download. When omitted, downloads every posterior of the current record version (~475 MB — dominated by the six full multivariate EIV posteriors at ~78–81 MB each; pass names= to fetch only what you need). Superseded initial-submission posteriors are pinned to the v0.2.0 record and must be requested by name.

  • cache_dir (Path or str, optional) – Destination directory. Defaults to the standard posterior cache.

  • force (bool) – Re-download files that already exist locally.

Returns:

Local paths of the downloaded .nc files.

Return type:

list of Path

Examples

Download the G₂/₃-only SST calibration (~78 MB):

>>> download_posteriors(["tx.GHEB.sst.sri03.G23"])

The four bundled posteriors (tx.GHEB.{sst,thm}.sri03.G23-N1p0 and tx.GHPU.{sst,thm}.sri03.p0) ship with the package and need no download; the Zenodo copies of the GHEB pair are the complete archival versions.

Download all#

download_all(cache_dir=None, data_dir=None, force=False, figures=False)[source]#

Download everything from Zenodo: forward posteriors + training data.

Files are downloaded individually; already-cached files are skipped unless force=True. With the default figures=False the total is ~510 MB: ~475 MB for the nine current-record forward posteriors (almost all of it the six full multivariate EIV posteriors, ~78–81 MB each) plus ~37 MB of training data (download_training_data()). The superseded v0.2.0 posteriors are not included; request them by name with download_posteriors().

Parameters:
  • cache_dir (Path or str, optional) – Destination for .nc posteriors. Defaults to the standard posterior cache directory.

  • data_dir (Path or str, optional) – Destination for training data files. Defaults to data/spreadsheets/.

  • force (bool) – Re-download files that already exist locally.

  • figures (bool) – Also fetch what the notebooks/manuscripts/SI_code* figure notebooks need beyond the default forward-posterior + training-CSV set: the invT reconstructions behind SI_code02/SI_code03 (download_invT_posteriors()) and the WOA23 climatologies SI_code00 reads (download_woa23(), >1 GB, skipped unless this is set). Does not yet cover every external file those notebooks read — see docs/ for the current gaps.

Return type:

None

Download training data#

download_training_data(dest_dir=None, force=False)[source]#

Download GDGT training data files from Zenodo.

Downloads the coretop/culture/mesocosm training CSVs, the CMEMS NO₃ uncertainty field used in the EIV calibration, and the ocean_prop_ds gridded ocean properties (CMEMS nitrate, WOA23 temperature). The CSVs and the CMEMS field are needed only to re-run the SI preprocessing and calibration notebooks from scratch and are NOT required for inverse temperature reconstructions — use download_posteriors() for that. ocean_prop_ds is the exception: it is also used at inference time by predict_T_from_proxyObs() for the optional site_lat/site_lon NO₃ lookup — see download_ocean_properties() to fetch just that one file. It also isn’t hosted on the TEXAS record at all; it comes from the companion GRL paper’s Zenodo record (see TRAINING_DATA_REGISTRY["ocean_prop_ds"]["record"]).

Parameters:
  • dest_dir (Path or str, optional) – Destination directory. Defaults to data/spreadsheets/ in the repo (or ~/.texas/data/spreadsheets/ when pip-installed).

  • force (bool) – Re-download files that already exist locally.

Returns:

Local paths of the downloaded files.

Return type:

list of Path

List posteriors#

list_posteriors(model_type='both', cache_dir=None)[source]#

List available posterior files in the cache directory.

Prints a summary and returns a dict of stem names that can be passed directly to predict_T_from_proxyObs(fwd_posterior=...).

Parameters:
  • model_type ("forward", "invT", or "both") – Which cache to inspect. Default "both".

  • cache_dir (Path or str, optional) – Override the default cache root. When given, both forward and invT subdirectories are looked for under this path.

Returns:

{"forward": [...], "invT": [...]} — lists of stem names (no .nc).

Return type:

dict

Set cache directory#

set_cache_dir(path)[source]#

Override TEXAS cache directories at runtime.

Call this before any posterior I/O. For a persistent override, set the TEXAS_CACHE_DIR environment variable instead.

Parameters:

path (str | Path) – Root directory for all TEXAS caches. Three subdirectories will be used inside it: TEXAS_posterior_cache/, TEXAS_invT_posterior_cache/ and TEXAS_kriged_grids_cache/.

Return type:

None


Screening#

Screen a record#

Flags samples lying outside the TEX₈₆–Scaled RI domain the calibration was trained on, using the same 90 % χ² Mahalanobis ellipse used to build the training set. A flagged sample is an extrapolation, not necessarily a bad measurement — the recommendation is to flag and interpret with caution rather than exclude.

The ellipse is a fixed property of the published calibration (the core-top reference cluster of Section 5.1), so nothing needs fitting — an unfitted detector on the standard features adopts it automatically:

from TEXAS.data import MahalanobisOutlierDetector

detector = MahalanobisOutlierDetector(["TEX86", "scaledRI_cren3"], confidence=0.90)
df["flagged"] = detector.detect_outliers_manual(df)

detect_outliers_manual is the manuscript’s criterion: the ellipse plus the warm-end exception, so samples with TEX₈₆ > 0.75 and Scaled RI₀₋₃ > 0.75 are retained rather than flagged — the calibration retains them too (Section 5.1, Fig. 4). detect_outliers applies the bare ellipse and would flag warm samples the calibration actually covers.

Do not call fit() or fit_predict() on your own record: that re-centres the ellipse onto the record, so the more unusual your data the less it flags.

class MahalanobisOutlierDetector(features, confidence=0.9, method='chi2', use_pinv_if_singular=True)[source]#

Fit Mahalanobis distance parameters on training data, apply to any dataset.

Examples

>>> # Fit on coretop data
>>> detector = MahalanobisOutlierDetector(['TEX86', 'ringIndex'])  # default confidence=0.90
>>> detector.fit(coretop_df)
>>> coretop_df['mahal_dist'] = detector.transform(coretop_df)
>>> coretop_df['outliers'] = detector.detect_outliers(coretop_df)
>>>
>>> # Apply to downcore data
>>> downcore_df['mahal_dist'] = detector.transform(downcore_df)
>>> downcore_df['outliers'] = detector.detect_outliers(downcore_df)
Parameters:
  • features (list[str])

  • confidence (float)

  • method (Literal['chi2', 'chi2_rounddown'])

  • use_pinv_if_singular (bool)

classmethod from_calibration(confidence=0.9)[source]#

Return a detector fixed to TEXAS’s published calibration domain.

Explicit form of the default behavior: a detector on the standard features screens against this domain without being fitted at all.

The ellipse is a property of the calibration – paper Section 5.1 fits it to core-top samples with GDGT-2/GDGT-3 <= 5 – not of the data being screened. Fitting on your own record instead re-centres the ellipse onto that record, so the more unusual the record the less it flags: an all-warm Paleogene section would report nothing.

Examples

>>> detector = MahalanobisOutlierDetector.from_calibration()
>>> df["flagged"] = detector.detect_outliers(df)
Parameters:

confidence (float)

Return type:

MahalanobisOutlierDetector

fit(df, *, columns=None, on_unscorable='warn')[source]#

Fit mean, covariance, and threshold on training data.

Parameters:
  • df (pd.DataFrame) – Training data

  • columns (dict, optional) – Mapping {logical_name: physical_column} for callers whose DataFrame uses different column names than self.features.

  • on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.

Returns:

self – Fitted detector instance

Return type:

MahalanobisOutlierDetector

transform(df, col_name=None, *, columns=None, on_unscorable='warn')[source]#

Compute Mahalanobis distances using fitted parameters.

Parameters:
  • df (pd.DataFrame) – Data to transform

  • col_name (str, optional) – If provided, add distances to df as this column

  • columns (dict, optional) – Mapping {logical_name: physical_column}.

  • on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.

Returns:

distances – Mahalanobis distances

Return type:

pd.Series

detect_outliers(df, col_name=None, *, columns=None, on_unscorable='warn')[source]#

Detect outliers using fitted threshold.

Parameters:
  • df (pd.DataFrame) – Data to screen

  • col_name (str, optional) – If provided, add outlier flags to df as this column

  • columns (dict, optional) – Mapping {logical_name: physical_column}.

  • on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.

Returns:

outliers – Boolean series (True=outlier, False=inlier, NaN=invalid)

Return type:

pd.Series

detect_outliers_manual(df, col_name=None, exclude_condition=None, *, columns=None, on_unscorable='warn')[source]#

Detect outliers using the paper’s screening criterion.

This is the recommended call for screening a record. It applies the calibration-domain ellipse and then restores the warm-end exception: samples with TEX86 > 0.75 and Scaled RI(0-3) > 0.75 are retained, not flagged, because the calibration itself retains them (paper Section 5.1 and the Fig. 4 caption: “data with TEX86 and Scaled RI above 0.75 are retained in the calibration dataset”). Screening with the bare ellipse instead (detect_outliers()) would flag warm samples the calibration actually covers.

Needs no fit(): on the standard features the published calibration domain is adopted automatically.

Parameters:
  • df (pd.DataFrame) – Data to screen

  • col_name (str, optional) – If provided, add manual outlier flags to df as this column

  • exclude_condition (pd.Series, optional) – Boolean series indicating samples to exclude from outlier detection. If None, applies default: (ringIndex > 3) & (TEX86 > 0.7)

  • columns (dict, optional) – Mapping {logical_name: physical_column} for callers whose DataFrame uses different column names than self.features.

  • on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.

Returns:

manual_outliers – Boolean series with manual exceptions applied

Return type:

pd.Series

Examples

>>> # Use default exception
>>> outliers = detector.detect_outliers_manual(df)
>>>
>>> # Custom exception
>>> custom_exclude = (df['SST'] > 30) & (df['TEX86'] > 0.8)
>>> outliers = detector.detect_outliers_manual(df, exclude_condition=custom_exclude)
fit_predict(df, col_name=None, *, columns=None, on_unscorable='warn')[source]#

Fit on df and flag its rows in one call (sklearn-style).

Equivalent to fit(df) followed by detect_outliers(df).

Warning

This fits the ellipse on df itself, which is right when df is a reference/training set and wrong for screening a record against the calibration – the domain would move with the data, so an unusual record flags less rather than more. To screen your own data, just call detect_outliers(): an unfitted detector on the standard features uses TEXAS’s published calibration domain automatically.

Parameters:
  • df (pd.DataFrame) – Data to fit on and screen

  • col_name (str, optional) – If provided, add outlier flags to df as this column

  • columns (dict, optional) – Mapping {logical_name: physical_column}.

  • on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.

Returns:

outliers – Boolean series (True=outlier, False=inlier, NaN=invalid)

Return type:

pd.Series

fit_transform(df, dist_col=None, outlier_col=None, manual_outlier_col=None, *, columns=None, on_unscorable='warn')[source]#

Fit and transform in one step.

Parameters:
  • df (pd.DataFrame) – Training data

  • dist_col (str, optional) – Column name for distances

  • outlier_col (str, optional) – Column name for outlier flags

  • manual_outlier_col (str, optional) – Column name for manual outlier flags

  • columns (dict, optional) – Mapping {logical_name: physical_column}.

  • on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.

Returns:

results – Dictionary containing: - ‘distances’: Mahalanobis distances - ‘outliers’: Outlier flags - ‘manual_outliers’: Manual outlier flags - ‘threshold’: Computed threshold

Return type:

dict

get_params()[source]#

Get fitted parameters.

Returns:

params – Dictionary of fitted parameters

Return type:

dict


Data builders#

Build forward data#

build_fwd_data(*, t_cul=None, proxy_cul=None, t_meso=None, proxy_meso=None, t_crtp=None, proxy_crtp=None, gdgt23ratio_crtp=None, sd_gdgt23ratio_crtp=None, no3_crtp=None, sd_no3_crtp=None, no3_cutoff=None, proxy_residuals_crtp=None, sd_proxyObs=None, R2_thermal=None, culmeso_posterior=None, prior_mean_t0=None, prior_sd_t0=None, prior_mean_k=None, prior_sd_k=None, prior_mean_b=None, prior_sd_b=None, prior_mean_v=None, prior_sd_v=None)[source]#

Build the Stan data dictionary for forward calibration models.

What to pass for each shipped forward model:
  • gen_logi_fixed_culmeso (stage 1) : t_cul, proxy_cul, t_meso, proxy_meso

  • …_hier_crtp_univ_priorApproxt_crtp, proxy_crtp + culmeso_posterior

    (temperature only, no predictors)

  • …_hier_crtp_multiv_priorApprox_eiv_t0shift
    : as above, plus gdgt23ratio_crtp, no3_crtp,

    their per-site SEs, sd_proxyObs and R2_thermal

Archived models (e.g. culmesocore) take the same keyword arguments.

Parameters:
  • t_cul – Culture temperature and proxy arrays.

  • proxy_cul – Culture temperature and proxy arrays.

  • t_meso – Mesocosm temperature and proxy arrays.

  • proxy_meso – Mesocosm temperature and proxy arrays.

  • t_crtp – Coretop temperature and proxy arrays.

  • proxy_crtp – Coretop temperature and proxy arrays.

  • gdgt23ratio_crtp – GDGT-2/GDGT-3 ratio for coretop samples. Sets use_gdgt23ratio=1 if non-zero/non-NaN.

  • sd_gdgt23ratio_crtp – Per-site measurement SE of gdgt23ratio (same units, linear). Required for the _eiv model; always included in the data dict (defaults to zeros when not provided, which disables the G₂/₃ EIV measurement model).

  • no3_crtp – Nitrate concentration for coretop samples. Sets use_no3=1 if non-zero/non-NaN.

  • sd_no3_crtp – Per-site measurement SE of NO₃ (μmol/L, linear space). Required for the _eiv model. Always included (defaults to zeros; sites with sd=0 receive only the lognormal prior and skip the normal measurement model).

  • no3_cutoff (float | None) – NO3 threshold for the nonthermal correction. Priority: (1) this arg, (2) culmeso_posterior attrs, (3) auto-calculated via Spearman method.

  • proxy_residuals_crtp (ndarray | None) – Pre-computed proxy residuals for NO3 threshold calculation. If omitted, residuals are computed internally by fitting a generalized logistic curve. Warning: Stan models use generalized logistic — residuals from other functional forms may shift the threshold.

  • sd_proxyObs – Per-site analytical SE of the proxy (Scaled RI units). Used by the _eiv model; defaults to 0.03 for every site (Schouten et al. 2013).

  • R2_thermal (float | None) – R² of a temperature-only coretop fit. Required by the _eiv model, where it scales the process-noise prior; omitted from the dict when not given.

  • culmeso_posterior (Dataset | None) – xr.Dataset from a completed culmeso forward run. Auto-extracts prior_mean_*/prior_sd_* hyperpriors and no3_cutoff (if saved in attrs).

  • prior_mean_*/prior_sd_* – Manual hyperprior values. Override auto-extracted values from culmeso_posterior for individual params.

Returns:

Stan-ready data dict with proxyObs_* keys, N_* counts, use_* flags,

and hyperpriors — ready for get_posterior().

Return type:

dict

Build invT input data#

build_invT_inputData(proxyObs, prior_mu_t, prior_sigma_t, *, fwd_posterior_name=None, predictors=None, config=None, fwd_posterior=None, fwd_cache_dir=None)[source]#

Build the data dictionary for Stan’s inverse model and sampler configuration.

WORKFLOW: ───────── 1. Load forward calibration posterior from .nc file (or accept a pre-loaded Dataset) 2. Randomly sample M parameter sets from that posterior 3. Extract calibration curve parameters (t0, k, b, v, sigma) 4. Package optional environmental predictors (GDGT-2/3, NO3) if used 5. Return data dict (for Stan) + sampler_kwargs (for CmdStanPy)

Parameters:
  • proxyObs (ndarray | List[float]) – Observed proxy values to predict temperature from (length N). Any proxy is accepted: scaledRI, TEX86, ringIndex, etc.

  • prior_mu_t (ndarray | float) – Prior mean temperature (scalar or array of length N)

  • prior_sigma_t (float) – Prior temperature uncertainty (e.g., 10°C)

  • fwd_posterior_name (str | None) – Name of saved forward calibration (without .nc extension). Not required when fwd_posterior is supplied directly.

  • predictors (Dict[str, ndarray] | None) – Optional environmental covariates {‘gdgt23ratio’: array, ‘no3’: array}

  • config (InvTConfig | None) – Configuration object controlling M, seed, etc.

  • fwd_posterior (Dataset | None) – Pre-loaded forward posterior xr.Dataset. When provided, fwd_posterior_name is ignored and no file I/O is performed. Useful when running from Google Colab or any pip-install context where the posterior cache is not available.

  • fwd_cache_dir (str | Path | None) – Directory to resolve fwd_posterior_name in. Defaults to the standard forward posterior cache. Use this to read posteriors from a project-local bundle without copying them into the cache.

Returns:

Dictionary for Stan’s data block sampler_kwargs: Dictionary for CmdStanPy sampling configuration

Return type:

data

InvT configuration#

class InvTConfig(mode='ensemble', n_draws=None, seed=42, no3_cutoff=0.0, suffix=None)[source]#

Knobs for the forward-to-inverse bridge in build_invT_inputData.

Controls how many parameter sets M are drawn from the forward posterior and marginalised over. The marginal-likelihood error falls as 1/sqrt(M) while cost rises linearly with it, so M is the main accuracy-versus-runtime dial of a reconstruction.

Parameters:
  • mode (str)

  • n_draws (int | None)

  • seed (int)

  • no3_cutoff (float | None)

  • suffix (str | None)

mode#

Must be "ensemble" (the only supported mode; kept for backward compatibility).

Type:

str

n_draws#

M, the number of forward draws. None auto-selects min(500, max(100, available_draws // 4)) – for a typical 4-chain x 1000-draw forward fit (4000 draws) this resolves to 500. The paper used this default (M = 500) for its downcore reconstructions and n_draws=300 (~5.8% Monte Carlo error) for its global coretop analyses; pass a value explicitly to trade accuracy for speed.

Type:

int | None

seed#

Seed for selecting those M draws, so a reconstruction is reproducible.

Type:

int

no3_cutoff#

Fallback NO3 cutoff (umol/L), used only when the forward posterior carries no no3_cutoff attr of its own – the posterior always wins.

Type:

float | None

suffix#

Force a forward parameter suffix (e.g. "crtp") instead of taking the highest-priority one present.

Type:

str | None

Modern NO₃ lookup#

lookup_modern_no3(lat, lon, ocean_dataset, variable='no3_sf2tc_avg', method='linear', *, lat_name=None, lon_name=None)[source]#

Look up modern NO₃ at one or more lat/lon coordinates from the ocean_prop_ds ocean-properties Dataset.

The dataset is typically the preprocessed ocean_prop_ds generated in SI_code00 (or fetched by get_ocean_prop_ds()), which contains the thermocline-depth-integrated CMEMS nitrate climatology on the regular WOA23 (lat, lon) grid. The returned value(s) are time-invariant (climatological mean) and intended as a modern-ocean proxy for the NO₃ correction in paleo reconstructions.

Parameters:
  • lat (float or array-like) – Latitude(s) in decimal degrees (−90 to 90). Pass a scalar for a single drill site; pass an array of length N to match N observations.

  • lon (float or array-like) – Longitude(s) in decimal degrees. Both −180–180 and 0–360 conventions are accepted — the function normalizes to match the dataset’s convention automatically.

  • ocean_dataset (xr.Dataset) – The ocean_prop_ds field (or any dataset) on a regular latitude/longitude grid containing variable. The axis names are detected (lat/lon, latitude/longitude, LAT/LON, …); pass lat_name / lon_name to override.

  • variable (str) – Name of the NO₃ variable to extract. Default "no3_sf2tc_avg" (thermocline depth-integrated CMEMS annual mean from SI_code00).

  • method ({"linear", "nearest"}) – Interpolation method. "linear" (default) performs bilinear interpolation and is preferred for smooth fields. "nearest" snaps to the closest grid cell and is useful when the dataset is sparse or has NaN-masked shelves.

  • lat_name (str, optional) – Latitude coordinate name. Auto-detected when omitted.

  • lon_name (str, optional) – Longitude coordinate name. Auto-detected when omitted.

Returns:

NO₃ value(s) in µmol/L. Shape matches the scalar/array input: a 0-d array for scalar inputs, 1-d array of length N for array inputs. NaN is returned for locations outside the dataset’s valid range (e.g. continental shelves masked in the CMEMS product).

Return type:

np.ndarray

Raises:
  • KeyError – If variable is not found in ocean_dataset.

  • ValueError – If the latitude/longitude axes cannot be resolved, or the resolved names are coordinates but not dimensions.

Examples

Single drill site:

>>> no3_val = lookup_modern_no3(15.3, -23.7, ocean_prop_ds)
>>> # returns scalar-equivalent float; broadcasts to all N obs automatically
>>> result = predict_T_from_proxyObs(..., no3=no3_val)

Multi-site stack (per-obs lookup):

>>> no3_arr = lookup_modern_no3(core_df["lat"].values,
...                              core_df["lon"].values,
...                              ocean_prop_ds)
>>> result = predict_T_from_proxyObs(..., no3=no3_arr)
lookup_no3_from_woa(lat, lon, woa_dataset, variable='no3_sf2tc_avg', method='linear', *, lat_name=None, lon_name=None)[source]#

Deprecated alias of lookup_modern_no3().

The name was wrong: the default no3_sf2tc_avg variable is the CMEMS nitrate climatology, not WOA23 (see the module docstring). Kept so existing calls keep working; emits a DeprecationWarning.

Parameters:
  • lat (float | ndarray)

  • lon (float | ndarray)

  • woa_dataset (Dataset)

  • variable (str)

  • method (Literal['linear', 'nearest'])

  • lat_name (str | None)

  • lon_name (str | None)

Return type:

ndarray


Forward calibration#

Get posterior#

get_posterior(data, stan_file, temptype, proxy_name, *, iter_warmup=None, iter_sampling=None, threads_per_chain=None, chains=None, parallel_chains=None, adapt_delta=None, max_treedepth=None, **kwargs)[source]#

Run a forward calibration in Stan and return the posterior.

Wraps StanSampler with automatic predictor detection, CPU configuration and metadata attachment. The returned dataset can be passed straight to predict_proxy_from_T or saved with save_posterior.

Parameters:
  • data (dict) – Stan data dict from build_fwd_data(). The use_* predictor flags are auto-detected from the arrays present.

  • stan_file (str) – Model name without .stan, e.g. "gen_logi_fixed_hier_crtp_multiv_priorApprox_eiv".

  • temptype (str) – Temperature target, e.g. "SST" or "thermoT".

  • proxy_name (str) – Proxy label, e.g. "scaledRI_cren3". Required: it is written to the attrs and validated when the posterior is later used for a reconstruction.

  • iter_warmup (int | None) – Warmup iterations per chain. Default 1000 (CmdStan’s).

  • iter_sampling (int | None) – Sampling iterations per chain. Default 1000.

  • threads_per_chain (int | None) – Threads per chain for reduce_sum models; auto-enabled for models whose filename contains reduce_sum.

  • chains (int | None) – Independent chains. Default 4.

  • parallel_chains (int | None) – Chains run at once. Auto-detected from the CPU count.

  • adapt_delta (float | None) – Target acceptance rate. Default 0.8; raise toward 0.99 to trade leapfrog steps for fewer divergences.

  • max_treedepth (int | None) – HMC maximum tree depth. Default 10.

  • **kwargs – Forwarded to CmdStanModel.sample.

Returns:

an xr.Dataset of draws with metadata attrs (model name, temptype, proxy_name, priors, stan_diag_*), and the human-readable sampler diagnostic summary.

Return type:

(posterior, diagnostics)

Raises:

ValueError – if active predictors are present but a univariate model was requested, or if an EIV model is requested without R2_thermal.

Example

>>> posterior, diag = get_posterior(
...     data, stan_file="gen_logi_fixed_hier_crtp_univ_priorApprox",
...     temptype="SST", proxy_name="scaledRI_cren3")
>>> save_posterior(posterior)

Save posterior#

save_posterior(posterior, cache_dir=None, overwrite=True, filename_suffix='', layout='auto', run='auto')[source]#

Save a forward-model posterior to disk as compressed NetCDF.

The filename is auto-generated from the posterior’s metadata attrs: {model}_{temptype}[_gdgt23ratio][_no3_{cutoff}][_{proxy_name}]{suffix}.nc

Parameters:
  • posterior (xr.Dataset) – Forward calibration posterior returned by get_posterior(). Must have stan_model_name, temptype, and proxy_name attrs set (proxy_name is required — a warning is raised if missing).

  • cache_dir (str or Path, optional) – Directory to write the file. Defaults to the standard forward posterior cache (data/cache/TEXAS_posterior_cache/ for source installs, ~/.texas/cache/TEXAS_posterior_cache/ for pip installs).

  • overwrite (bool) – If False, raise FileExistsError when the output path already exists. Default True.

  • filename_suffix (str, optional) – Extra tag appended before .nc under the legacy layout, and the run/member token under the case layout. Leading/trailing underscores are stripped. Do not pass a date here. Filenames do not carry date stamps; the run date is recorded in the run_timestamp attr, where it survives a rename and does not have to be parsed back out of a path. Prefer run= for an explicit member.

  • run (str or int, optional) – Optional run/member token under the case layout. "auto" (default) writes no member, so a configuration has one canonical path and a re-run replaces it; pass a value only to keep a run deliberately apart. An explicit run wins over filename_suffix for the case path. With overwrite=False an existing file raises FileExistsError.

  • layout ({"auto", "case", "legacy"}) – Where to write. "case" uses the flat CESM-style case name (tx.GHEB.sst.sri03.G23-N1p0.fwd.nc); "legacy" uses the historical long flat filename; "auto" (default) prefers the case layout and falls back to legacy with a warning if no case id can be derived. See TEXAS.utils.naming.

Returns:

Absolute path of the saved .nc file.

Return type:

Path

Load posterior#

load_posterior(model_name, model_type='forward', cache_dir=None)[source]#

Load a posterior from disk: {model_name}.nc in the appropriate cache directory.

Parameters:
  • model_name (str) – Name of the model file (without .nc extension)

  • model_type (Literal['forward', 'invT']) – Type of posterior (“forward” or “invT”)

  • cache_dir (str | Path | None) – Custom cache directory (overrides default locations)

Returns:

xarray.Dataset containing the posterior

Raises:

FileNotFoundError – If the posterior file doesn’t exist

Return type:

Dataset


Ensemble#

Generate ensemble (auto)#

generate_ensemble_auto(post_ds, x_vals, model_type='auto', gdgt23ratio=None, no3=None, no3_cutoff=None, return_full_ensemble=False, suffix=None, **kwargs)[source]#

Sample draws from a forward posterior and compute calibration-curve percentiles.

Inspects the posterior’s stan_model_name attr and data_vars to determine the model function, parameter names, and optional-predictor flags automatically, then delegates to generate_ensemble.

Parameters:
  • post_ds (xr.Dataset) – Forward calibration posterior returned by get_posterior() or loaded with load_posterior().

  • x_vals (np.ndarray) – Temperature values (°C) at which to evaluate the calibration curve.

  • model_type ({"auto", "forward", "inverse"}) – Force forward or inverse dispatch; "auto" (default) infers from the posterior. InvT posteriors are not supported — use predict_T_from_proxyObs() instead.

  • gdgt23ratio (np.ndarray, optional) – GDGT-2/3 ratio values; required when the posterior was fitted with a multivariate (GDGT-2/3) model.

  • no3 (np.ndarray, optional) – NO₃ concentrations (µmol/L); required when the posterior uses the NO₃ correction.

  • no3_cutoff (float, optional) – Override the NO₃ cutoff from the posterior attrs.

  • return_full_ensemble (bool) – If True, return the full M × N draw matrix in addition to percentiles. Default False.

  • suffix (str, optional) – Force a specific parameter suffix (e.g. "crtp"); overrides auto-detection.

  • **kwargs – Forwarded to generate_ensemble.

Returns:

Keys "p1" … "p99" (and optionally "ensemble") — each a numpy array of length len(x_vals).

Return type:

dict

Raises:

NotImplementedError – If called with an invT posterior.

Detect model and params#

detect_model_and_params(posterior_ds, suffix=None)[source]#

Infer the curve, parameter names and predictor flags from a forward posterior.

Structure is read from the posterior’s data_vars and attrs, never from stan_model_name, so a posterior that was renamed or downloaded from Zenodo still dispatches correctly. The presence of a v parameter selects the generalized curve over the plain logistic; gamma_G23* / gamma_NO3* select the T0-shift parameterization over the additive one.

Parameters:
  • posterior_ds (Dataset) – A forward calibration posterior.

  • suffix (str) – Force a parameter suffix (e.g. "crtp"). None selects by the priority order crtp > culmesocore > culmeso > meso > cul.

Returns:

A dict with model_function (the callable to evaluate), param_names (unsuffixed names, in the order the callable wants them), suffix, is_multivariate, use_gdgt23ratio, use_no3 and no3_cutoff.

Raises:

ValueError – if gamma coefficients are present without a v parameter. No shipped model produces that combination, and fitting the additive form to gamma coefficients would be wrong by a whole parameterization while still looking plausible.


Diagnostics#

Sampler diagnostics#

summarize_sampler_diagnostics(fit)[source]#

Summarize a CmdStanPy fit’s convergence diagnostics as stan_diag_* attrs.

Reads divergent transitions, max-treedepth saturation, E-BFMI, R-hat and bulk ESS, and grades each against a fixed threshold, so a run can be accepted or rejected without re-opening the raw fit. Attached to every posterior by StanSampler.sample and get_invT_posterior.

Parameters:

fit – A CmdStanMCMC object, as returned by CmdStanModel.sample.

Returns:

the counts and percentages (n_divergent, pct_divergent, n_max_treedepth, pct_max_treedepth, min_ebfmi, max_rhat, n_high_rhat, min_ess_bulk), a *_status of "PASS", "FAIL" or "UNKNOWN" beside each, and stan_diag_overall_status. The thresholds are: divergences < 1%, max-treedepth < 5%, R-hat < 1.01, bulk ESS > 100. All values are plain Python scalars, so the dict can be written straight into NetCDF attrs.

Return type:

A flat dict of stan_diag_* entries

Summary table#

create_summary_table(datasets)[source]#

Tabulate the stan_diag_* attrs of several posteriors, one row each.

Parameters:

datasets (list) – Posteriors carrying stan_diag_* attrs, as written by summarize_sampler_diagnostics().

Returns:

A DataFrame with a model column (each dataset’s filename attr, or "unknown") plus one column per diagnostic, with the stan_diag_ prefix stripped. A dataset missing a diagnostic gets NaN in that column rather than being dropped.

Return type:

DataFrame


Plotting#

Plot prior distributions#

plot_prior_distributions(priors_list=None, posterior_datasets=None, posterior_labels_list=None, show_suptitle=True, kde_bw=0.3, focus_on_posterior=True, include_groups=('t0', 'k', 'b', 'v', 'a', 'beta_G23', 'beta_NO3', 'gamma_G23', 'gamma_NO3', 'betaLogit_G23', 'betaLogit_NO3', 'sigma_proxyObs'), suffix_include=None, zoomin_suffix=None, zoomin_dataset_idx=None, use_linestyle_by_param=False, show_histogram=True, show_annotation=False, set_linewidth=1.5, set_fig_width_factor=3, set_fig_height_factor=3.5, set_leg_max_ncol=3, color_list=None, param_source_map=None, annotation_style='ci95', show_subplot_legend=True, show_figure_legend=True, show_prior_expression=True, cache_dir=None)[source]#

Plot priors + any number of posterior distributions in a grid, split by parameter group (t0, k, b, etc.).

Parameters:
  • param_source_map (Dict[str, int] | None) –

    Optional dict mapping a param group name to the index of the dataset in posterior_datasets that should be used as the sole source for that group. All other datasets are skipped for that group.

    Use this when different parameters come from different posteriors — e.g. logistic params (t0, k, b…) from a culmeso run and beta coefficients from a multivariate crtp run:

    plot_prior_distributions(
        posterior_datasets=[culmeso_ds, crtp_multiv_ds],
        param_source_map={"beta_G23": 1, "beta_NO3": 1},
    )
    

    When a group is not in param_source_map, all datasets are searched as usual.

  • include_groups (Sequence[str]) – Parameter-group prefixes to draw, in panel order. The default covers the thermal parameters plus every coefficient naming scheme in PREDICTOR_GROUPS (beta_* for the parent model, gamma_* for t0shift, betaLogit_* for boundedCeil), so the same call works for any of them and for figures mixing several. A group with no matching variable in any dataset is skipped, as is a coefficient group whose predictor no dataset switched on.

  • cache_dir (str | Path | None) – Directory to resolve any name strings in posterior_datasets. Defaults to the standard forward posterior cache. Loaded Datasets are passed through untouched.

  • priors_list (List[str] | Dict[str, str] | None)

  • posterior_datasets (List[xr.Dataset] | None)

  • posterior_labels_list (List[str] | None)

  • show_suptitle (bool)

  • kde_bw (float)

  • focus_on_posterior (bool)

  • suffix_include (List[str] | None)

  • zoomin_suffix (str | List[str] | None)

  • zoomin_dataset_idx (int | None)

  • use_linestyle_by_param (bool)

  • show_histogram (bool)

  • show_annotation (bool)

  • set_linewidth (float)

  • set_fig_width_factor (float)

  • set_fig_height_factor (float)

  • set_leg_max_ncol (int)

  • color_list (Sequence[str] | None)

  • annotation_style (Literal['ci95', 'ci68', 'sigma'])

  • show_subplot_legend (bool)

  • show_figure_legend (bool)

  • show_prior_expression (bool)