API Reference#
Quick reference#
Function |
Description |
|---|---|
Compute Scaled RI (RI₀₋₃ by default) from six isoGDGT abundances |
|
Forward: temperature → proxy percentiles (pure Python) |
|
Inverse: proxy → temperature with full uncertainty (runs Stan) |
|
Flag reconstructed samples the calibration cannot support |
|
Download forward posteriors from Zenodo |
|
Download training CSVs, CMEMS NO₃ uncertainty field + |
|
Print and return posterior names in the local cache and bundled with the package |
|
Flag samples outside the published calibration domain (no fitting required) |
|
Control the number of forward draws M a reconstruction marginalises over |
|
Build validated Stan data dict for forward calibration |
|
Run forward calibration Stan sampling |
|
Persist forward posterior as compressed NetCDF |
|
Load a forward or invT posterior from the cache |
|
Divergences, R-hat, ESS, E-BFMI |
Prediction#
Compute Scaled Ring Index#
- compute_scaledRI(gdgt0, gdgt1, gdgt2, gdgt3, cren, cren_prime, *, cren_weight=3, cren_rings=None)[source]#
Compute Scaled Ring Index from six isoGDGT abundances.
Accepts raw LC/MS peak areas or fractional abundances — both give identical results because the formula divides by the total sum of all six GDGTs, so any common scale factor drops out. Default
cren_weight=3produces scaledRI_cren3 (RI₀₋₃), the canonical proxy used in TEXAS calibration posteriors.- Parameters:
gdgt0 (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).
gdgt1 (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).
gdgt2 (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).
gdgt3 (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).
cren (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).
cren_prime (float or array-like) – isoGDGT abundances — GDGT-0, GDGT-1, GDGT-2, GDGT-3, crenarchaeol, crenarchaeol regioisomer (cren’). Raw LC/MS peak areas and fractional abundances give the same result (see above).
cren_weight (float) – Weight carried by both crenarchaeol and its regioisomer, which also sets the normalization (see Notes).
3→ scaledRI_cren3 / RI₀₋₃ (default, recommended).4→ scaledRI / RI₀₋₄ (Zhang et al. 2016 convention).cren_rings (int, optional) – Deprecated alias for
cren_weight. Emits aDeprecationWarning.
- Returns:
Scaled Ring Index, dimensionless, nominally in [0, 1].
- Return type:
numpy.ndarray or float
Notes
The formula (paper Eqs. 6 and 8 for the default
cren_weight=3) is:RI = (1·GDGT1 + 2·GDGT2 + 3·GDGT3 + cren_weight·cren + cren_weight·cren') / (GDGT0 + GDGT1 + GDGT2 + GDGT3 + cren + cren') scaledRI = RI / cren_weightThe parameter does two jobs at once: it is the coefficient on cren and cren’ in the numerator and the constant the whole index is divided by, so it fixes the scale on which every sample is expressed, not just the two crenarchaeol terms. That second job is what makes a value chosen here part of the proxy’s definition rather than a detail — a posterior calibrated at 3 cannot read an index built at 4.
The older name
cren_ringsis still accepted, with a deprecation warning. It was misleading twice over: it named only the first job, and it implied a count of cyclic moieties. Neither 3 nor 4 is that count. They are calibration conventions, which is why both are offered and why neither is “correct”.Examples
>>> compute_scaledRI(0.45, 0.10, 0.08, 0.05, 0.30, 0.02) array(0.45666...)
>>> import pandas as pd >>> df = pd.read_csv("my_gdgt_data.csv") >>> df["scaledRI_cren3"] = compute_scaledRI( ... df["GDGT-0"], df["GDGT-1"], df["GDGT-2"], df["GDGT-3"], ... df["cren"], df["cren_prime"], ... )
Predict proxy from T#
- predict_proxy_from_T(temperatures, posterior, *, n_draws=500, percentiles=[5, 50, 95], return_full=False, seed=42, gdgt23ratio=None, no3=None, no3_cutoff=None, suffix=None, fwd_cache_dir=None)[source]#
Forward prediction: temperature → proxy percentiles (Scaled RI, TEX86, or any fitted proxy).
Samples n_draws self-consistent parameter sets from the forward calibration posterior (all parameters drawn from the same posterior index, preserving correlations) and evaluates the calibration curve at each requested temperature. This is the forward model of the paper: the generalized logistic curve of Eq. 9 and, for multivariate calibrations, the T₀ shift of Eq. 13 (older additive posteriors add their corrections to the curve instead).
- Parameters:
temperatures (array-like) – Temperatures (°C) at which to evaluate the calibration curve.
posterior (xr.Dataset or str) – Forward calibration posterior — either a loaded xr.Dataset or a saved-file name string (looked up in the posterior cache).
n_draws (int) – Number of posterior draws to sample. Default 500.
percentiles (list of float) – Percentiles to return, e.g. [5, 50, 95].
return_full (bool) – If True, also return the full (n_draws × len(temperatures)) ensemble array and run metadata under keys
"ensemble"and"metadata".seed (int) – Random seed for reproducible draw sampling.
gdgt23ratio (array-like, optional) – GDGT-2/GDGT-3 ratio values (one per temperature point). Required only when the posterior was fitted with a G₂/₃ term (γ_{G₂/₃} in T₀-shift posteriors, β_{G₂/₃} in older additive ones).
no3 (array-like, optional) – Nitrate concentration values (one per temperature point). Required only when the posterior was fitted with NO₃ correction.
no3_cutoff (float, optional) – Nitrate threshold (μmol/L) below which the NO₃ correction applies. Defaults to the value stored in the posterior attributes.
suffix (str, optional) – Force a specific parameter suffix (e.g.
"crtp"). Auto-detected by priority order when omitted.fwd_cache_dir (Path or str, optional) – Directory to resolve posterior in when it is given as a name string. Defaults to the standard forward posterior cache. Ignored when a loaded Dataset is passed.
- Returns:
"x_vals"— temperature array (°C)"pN"— one key per requested percentile, e.g."p5","p50","p95""ensemble"— full array, shape (n_draws, len(temperatures)), if return_full=True"metadata"— run metadata dict, if return_full=True- Return type:
dict with keys
Predict T from proxy observations#
- predict_T_from_proxyObs(proxyObs, prior_mu_t, prior_sigma_t, fwd_posterior=None, *, proxy_name=None, temptype=None, site_name=None, predictors=None, no3=None, gdgt23ratio=None, site_lat=None, site_lon=None, no3_dataset=None, no3_dataset_var='no3_sf2tc_avg', flags=True, tex86=None, config=None, chains=4, iter_warmup=500, iter_sampling=1000, seed=42, threads_per_chain=None, save_results=False, save_draws=False, filename_tag=None, cache_dir=None, fwd_cache_dir=None)[source]#
Reconstruct temperature from proxy observations, with full uncertainty.
Marginalises over M draws from the forward calibration posterior so calibration uncertainty propagates into the reconstruction. See TEXAS Proxy System Model — Bayesian Inference Workflow Section 8 for the default calibration, the NO3 resolution order, and what the quality flags mean.
- Parameters:
proxyObs (ndarray | List[float]) – Observed proxy values, shape (N,) – scaledRI, TEX86, …
prior_mu_t (ndarray | float) – Prior mean temperature (degC), scalar or shape (N,).
prior_sigma_t (float) – Prior temperature SD (degC); use ~10 when little is known.
fwd_posterior (str | Dataset | None) – Forward calibration: a case id/legacy name (str, loaded from the cache) or a pre-loaded
xr.Dataset. If omitted, a bundled calibration for temptype is used: the multivariate one (G23 + NO3) when any predictor is given, otherwise the temperature-only one (with a warning).proxy_name (str | None) – Proxy label. Inherited from the calibration when omitted.
temptype (str | None) –
"SST"or"thermoT"; a label, except it picks the default calibration when fwd_posterior is omitted.site_name (str | None) – Label for the metadata and output filenames.
predictors (Dict[str, ndarray] | None) – Non-thermal predictor arrays, e.g.
{"gdgt23ratio": ..., "no3": ...}.no3 (float | ndarray | None) – Nitrate (umol/L), scalar or shape (N,). Overrides predictors.
gdgt23ratio (float | ndarray | None) – GDGT-2/3 ratio, scalar or shape (N,). Overrides predictors.
site_lat (float | ndarray | None) – Latitude(s) for a modern NO3 lookup in the CMEMS nitrate climatology (
ocean_prop_ds). Needs site_lon.site_lon (float | ndarray | None) – Longitude(s) for the same lookup.
no3_dataset (Dataset | None) – The
ocean_prop_ds(lat, lon)field (CMEMS nitrate on the WOA23 grid); downloaded from Zenodo and cached if omitted.no3_dataset_var (str) – Variable to read from it. Default
"no3_sf2tc_avg".flags (bool) – Attach
result["flags"], one row per observation. Default True.tex86 (float | ndarray | None) – TEX86 for the same samples; only
outside_domainuses it.config (InvTConfig | None) –
InvTConfigcontrolling M, the seed and the suffix.chains (int) – MCMC chains. Default 4.
iter_warmup (int) – Warmup iterations per chain. Default 500.
iter_sampling (int) – Sampling iterations per chain. Default 1000.
seed (int) – Random seed. Default 42.
threads_per_chain (int | None) – Within-chain parallelism for
reduce_summodels.save_results (bool) – Write the quantile
.ncand results.npz.save_draws (bool) – Also write the raw draws as
{base}_draws.nc.filename_tag (str | Sequence[str] | None) – Extra tag(s) for the output filenames.
cache_dir (str | Path | None) – Where outputs are written. Defaults to the invT cache.
fwd_cache_dir (str | Path | None) – Where a named fwd_posterior is read from; a different directory from cache_dir.
- Returns:
A dict with
"proxyObs","proxy_name","metadata", one"pN"array per posterior quantile ("p5","p50", …), and"flags"when flags is True.- Return type:
Dict[str, Any]
Per-observation quality flags#
Returned automatically as result["flags"], and callable on its own against a saved reconstruction. What each column means, and how posterior_prior_width_ratio relates to the manuscript’s Resolved diagnostic, is on the PSM page.
- compute_quality_flags(proxyObs, fwd_posterior, *, predictors=None, prior_sigma_t=None, result=None, tex86=None, suffix=None, domain_confidence=0.9, prior_dominated_ratio=0.85)[source]#
Flag observations a reconstruction cannot support, one row per input.
- Parameters:
proxyObs (array-like, shape (N,)) – The observations handed to the inverse model.
fwd_posterior (xr.Dataset) – The forward calibration the reconstruction used. Both the parameter draws and the
attrsrecording the calibration’s own data range are read from it.predictors (dict, optional) –
{"gdgt23ratio": ..., "no3": ...}as passed to the reconstruction. Scalars are broadcast. Absent predictors that the calibration uses are reported bypredictor_missing.prior_sigma_t (float, optional) – The temperature prior SD. Required for
prior_dominated; without it that column ispd.NA.result (dict, optional) – The reconstruction result.
prior_dominatedneedsp16/p84from it; without it that column ispd.NA.tex86 (float or array-like, optional) – TEX86 for the same samples. The published calibration-domain ellipse is two-dimensional (TEX86 x Scaled RI), so
outside_domaincan only be evaluated when both are available; it ispd.NAotherwise.suffix (str, optional) – Parameter suffix to read (
"crtp","culmeso", …). Chosen by the usual priority when omitted.domain_confidence (float, default 0.90) – Confidence level of the Mahalanobis ellipse.
prior_dominated_ratio (float, default 0.85) – Posterior/prior SD ratio at or above which a row is prior-dominated.
- Returns:
N rows. Boolean columns from
FLAG_COLUMNS, graded columns fromDIAGNOSTIC_COLUMNS, andany_flag.A check that could not be evaluated is
pd.NA, neverFalse— “not assessed” and “passed” are different answers, and collapsing them would quietly launder the first into the second.any_flagtherefore ORs only what was actually evaluated, and only the defect columns:ADVISORY_COLUMNSare reported but do not vote.- Return type:
pd.DataFrame
Download and cache#
Download posteriors#
- download_posteriors(names=None, cache_dir=None, force=False)[source]#
Download forward calibration posteriors from Zenodo.
- Parameters:
names (list of str, optional) – Subset of
POSTERIOR_REGISTRYkeys to download. When omitted, downloads every posterior of the current record version (~475 MB — dominated by the six full multivariate EIV posteriors at ~78–81 MB each; passnames=to fetch only what you need). Superseded initial-submission posteriors are pinned to the v0.2.0 record and must be requested by name.cache_dir (Path or str, optional) – Destination directory. Defaults to the standard posterior cache.
force (bool) – Re-download files that already exist locally.
- Returns:
Local paths of the downloaded
.ncfiles.- Return type:
list of Path
Examples
Download the G₂/₃-only SST calibration (~78 MB):
>>> download_posteriors(["tx.GHEB.sst.sri03.G23"])
The four bundled posteriors (tx.GHEB.{sst,thm}.sri03.G23-N1p0 and tx.GHPU.{sst,thm}.sri03.p0) ship with the package and need no download; the Zenodo copies of the GHEB pair are the complete archival versions.
Download all#
- download_all(cache_dir=None, data_dir=None, force=False, figures=False)[source]#
Download everything from Zenodo: forward posteriors + training data.
Files are downloaded individually; already-cached files are skipped unless force=True. With the default
figures=Falsethe total is ~510 MB: ~475 MB for the nine current-record forward posteriors (almost all of it the six full multivariate EIV posteriors, ~78–81 MB each) plus ~37 MB of training data (download_training_data()). The superseded v0.2.0 posteriors are not included; request them by name withdownload_posteriors().- Parameters:
cache_dir (Path or str, optional) – Destination for
.ncposteriors. Defaults to the standard posterior cache directory.data_dir (Path or str, optional) – Destination for training data files. Defaults to
data/spreadsheets/.force (bool) – Re-download files that already exist locally.
figures (bool) – Also fetch what the
notebooks/manuscripts/SI_code*figure notebooks need beyond the default forward-posterior + training-CSV set: the invT reconstructions behind SI_code02/SI_code03 (download_invT_posteriors()) and the WOA23 climatologies SI_code00 reads (download_woa23(), >1 GB, skipped unless this is set). Does not yet cover every external file those notebooks read — seedocs/for the current gaps.
- Return type:
None
Download training data#
- download_training_data(dest_dir=None, force=False)[source]#
Download GDGT training data files from Zenodo.
Downloads the coretop/culture/mesocosm training CSVs, the CMEMS NO₃ uncertainty field used in the EIV calibration, and the
ocean_prop_dsgridded ocean properties (CMEMS nitrate, WOA23 temperature). The CSVs and the CMEMS field are needed only to re-run the SI preprocessing and calibration notebooks from scratch and are NOT required for inverse temperature reconstructions — usedownload_posteriors()for that.ocean_prop_dsis the exception: it is also used at inference time bypredict_T_from_proxyObs()for the optionalsite_lat/site_lonNO₃ lookup — seedownload_ocean_properties()to fetch just that one file. It also isn’t hosted on the TEXAS record at all; it comes from the companion GRL paper’s Zenodo record (seeTRAINING_DATA_REGISTRY["ocean_prop_ds"]["record"]).- Parameters:
dest_dir (Path or str, optional) – Destination directory. Defaults to
data/spreadsheets/in the repo (or~/.texas/data/spreadsheets/when pip-installed).force (bool) – Re-download files that already exist locally.
- Returns:
Local paths of the downloaded files.
- Return type:
list of Path
List posteriors#
- list_posteriors(model_type='both', cache_dir=None)[source]#
List available posterior files in the cache directory.
Prints a summary and returns a dict of stem names that can be passed directly to
predict_T_from_proxyObs(fwd_posterior=...).- Parameters:
model_type ("forward", "invT", or "both") – Which cache to inspect. Default
"both".cache_dir (Path or str, optional) – Override the default cache root. When given, both forward and invT subdirectories are looked for under this path.
- Returns:
{"forward": [...], "invT": [...]}— lists of stem names (no.nc).- Return type:
dict
Set cache directory#
- set_cache_dir(path)[source]#
Override TEXAS cache directories at runtime.
Call this before any posterior I/O. For a persistent override, set the
TEXAS_CACHE_DIRenvironment variable instead.- Parameters:
path (str | Path) – Root directory for all TEXAS caches. Three subdirectories will be used inside it:
TEXAS_posterior_cache/,TEXAS_invT_posterior_cache/andTEXAS_kriged_grids_cache/.- Return type:
None
Screening#
Screen a record#
Flags samples lying outside the TEX₈₆–Scaled RI domain the calibration was trained on, using the same 90 % χ² Mahalanobis ellipse used to build the training set. A flagged sample is an extrapolation, not necessarily a bad measurement — the recommendation is to flag and interpret with caution rather than exclude.
The ellipse is a fixed property of the published calibration (the core-top reference cluster of Section 5.1), so nothing needs fitting — an unfitted detector on the standard features adopts it automatically:
from TEXAS.data import MahalanobisOutlierDetector
detector = MahalanobisOutlierDetector(["TEX86", "scaledRI_cren3"], confidence=0.90)
df["flagged"] = detector.detect_outliers_manual(df)
detect_outliers_manual is the manuscript’s criterion: the ellipse plus the warm-end exception, so samples with TEX₈₆ > 0.75 and Scaled RI₀₋₃ > 0.75 are retained rather than flagged — the calibration retains them too (Section 5.1, Fig. 4). detect_outliers applies the bare ellipse and would flag warm samples the calibration actually covers.
Do not call fit() or fit_predict() on your own record: that re-centres the ellipse onto the record, so the more unusual your data the less it flags.
- class MahalanobisOutlierDetector(features, confidence=0.9, method='chi2', use_pinv_if_singular=True)[source]#
Fit Mahalanobis distance parameters on training data, apply to any dataset.
Examples
>>> # Fit on coretop data >>> detector = MahalanobisOutlierDetector(['TEX86', 'ringIndex']) # default confidence=0.90 >>> detector.fit(coretop_df) >>> coretop_df['mahal_dist'] = detector.transform(coretop_df) >>> coretop_df['outliers'] = detector.detect_outliers(coretop_df) >>> >>> # Apply to downcore data >>> downcore_df['mahal_dist'] = detector.transform(downcore_df) >>> downcore_df['outliers'] = detector.detect_outliers(downcore_df)
- Parameters:
features (list[str])
confidence (float)
method (Literal['chi2', 'chi2_rounddown'])
use_pinv_if_singular (bool)
- classmethod from_calibration(confidence=0.9)[source]#
Return a detector fixed to TEXAS’s published calibration domain.
Explicit form of the default behavior: a detector on the standard features screens against this domain without being fitted at all.
The ellipse is a property of the calibration – paper Section 5.1 fits it to core-top samples with GDGT-2/GDGT-3 <= 5 – not of the data being screened. Fitting on your own record instead re-centres the ellipse onto that record, so the more unusual the record the less it flags: an all-warm Paleogene section would report nothing.
Examples
>>> detector = MahalanobisOutlierDetector.from_calibration() >>> df["flagged"] = detector.detect_outliers(df)
- Parameters:
confidence (float)
- Return type:
- fit(df, *, columns=None, on_unscorable='warn')[source]#
Fit mean, covariance, and threshold on training data.
- Parameters:
df (pd.DataFrame) – Training data
columns (dict, optional) – Mapping {logical_name: physical_column} for callers whose DataFrame uses different column names than
self.features.on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.
- Returns:
self – Fitted detector instance
- Return type:
- transform(df, col_name=None, *, columns=None, on_unscorable='warn')[source]#
Compute Mahalanobis distances using fitted parameters.
- Parameters:
df (pd.DataFrame) – Data to transform
col_name (str, optional) – If provided, add distances to df as this column
columns (dict, optional) – Mapping {logical_name: physical_column}.
on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.
- Returns:
distances – Mahalanobis distances
- Return type:
pd.Series
- detect_outliers(df, col_name=None, *, columns=None, on_unscorable='warn')[source]#
Detect outliers using fitted threshold.
- Parameters:
df (pd.DataFrame) – Data to screen
col_name (str, optional) – If provided, add outlier flags to df as this column
columns (dict, optional) – Mapping {logical_name: physical_column}.
on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.
- Returns:
outliers – Boolean series (True=outlier, False=inlier, NaN=invalid)
- Return type:
pd.Series
- detect_outliers_manual(df, col_name=None, exclude_condition=None, *, columns=None, on_unscorable='warn')[source]#
Detect outliers using the paper’s screening criterion.
This is the recommended call for screening a record. It applies the calibration-domain ellipse and then restores the warm-end exception: samples with TEX86 > 0.75 and Scaled RI(0-3) > 0.75 are retained, not flagged, because the calibration itself retains them (paper Section 5.1 and the Fig. 4 caption: “data with TEX86 and Scaled RI above 0.75 are retained in the calibration dataset”). Screening with the bare ellipse instead (
detect_outliers()) would flag warm samples the calibration actually covers.Needs no
fit(): on the standard features the published calibration domain is adopted automatically.- Parameters:
df (pd.DataFrame) – Data to screen
col_name (str, optional) – If provided, add manual outlier flags to df as this column
exclude_condition (pd.Series, optional) – Boolean series indicating samples to exclude from outlier detection. If None, applies default: (ringIndex > 3) & (TEX86 > 0.7)
columns (dict, optional) – Mapping {logical_name: physical_column} for callers whose DataFrame uses different column names than
self.features.on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.
- Returns:
manual_outliers – Boolean series with manual exceptions applied
- Return type:
pd.Series
Examples
>>> # Use default exception >>> outliers = detector.detect_outliers_manual(df) >>> >>> # Custom exception >>> custom_exclude = (df['SST'] > 30) & (df['TEX86'] > 0.8) >>> outliers = detector.detect_outliers_manual(df, exclude_condition=custom_exclude)
- fit_predict(df, col_name=None, *, columns=None, on_unscorable='warn')[source]#
Fit on df and flag its rows in one call (sklearn-style).
Equivalent to
fit(df)followed bydetect_outliers(df).Warning
This fits the ellipse on df itself, which is right when df is a reference/training set and wrong for screening a record against the calibration – the domain would move with the data, so an unusual record flags less rather than more. To screen your own data, just call
detect_outliers(): an unfitted detector on the standard features uses TEXAS’s published calibration domain automatically.- Parameters:
df (pd.DataFrame) – Data to fit on and screen
col_name (str, optional) – If provided, add outlier flags to df as this column
columns (dict, optional) – Mapping {logical_name: physical_column}.
on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.
- Returns:
outliers – Boolean series (True=outlier, False=inlier, NaN=invalid)
- Return type:
pd.Series
- fit_transform(df, dist_col=None, outlier_col=None, manual_outlier_col=None, *, columns=None, on_unscorable='warn')[source]#
Fit and transform in one step.
- Parameters:
df (pd.DataFrame) – Training data
dist_col (str, optional) – Column name for distances
outlier_col (str, optional) – Column name for outlier flags
manual_outlier_col (str, optional) – Column name for manual outlier flags
columns (dict, optional) – Mapping {logical_name: physical_column}.
on_unscorable ({'warn', 'raise', 'ignore'}, default 'warn') – Policy for rows that map to a present column but are NaN/Inf.
- Returns:
results – Dictionary containing: - ‘distances’: Mahalanobis distances - ‘outliers’: Outlier flags - ‘manual_outliers’: Manual outlier flags - ‘threshold’: Computed threshold
- Return type:
dict
Data builders#
Build forward data#
- build_fwd_data(*, t_cul=None, proxy_cul=None, t_meso=None, proxy_meso=None, t_crtp=None, proxy_crtp=None, gdgt23ratio_crtp=None, sd_gdgt23ratio_crtp=None, no3_crtp=None, sd_no3_crtp=None, no3_cutoff=None, proxy_residuals_crtp=None, sd_proxyObs=None, R2_thermal=None, culmeso_posterior=None, prior_mean_t0=None, prior_sd_t0=None, prior_mean_k=None, prior_sd_k=None, prior_mean_b=None, prior_sd_b=None, prior_mean_v=None, prior_sd_v=None)[source]#
Build the Stan data dictionary for forward calibration models.
- What to pass for each shipped forward model:
gen_logi_fixed_culmeso (stage 1) : t_cul, proxy_cul, t_meso, proxy_meso
- …_hier_crtp_univ_priorApproxt_crtp, proxy_crtp + culmeso_posterior
(temperature only, no predictors)
- …_hier_crtp_multiv_priorApprox_eiv_t0shift
- : as above, plus gdgt23ratio_crtp, no3_crtp,
their per-site SEs, sd_proxyObs and R2_thermal
Archived models (e.g. culmesocore) take the same keyword arguments.
- Parameters:
t_cul – Culture temperature and proxy arrays.
proxy_cul – Culture temperature and proxy arrays.
t_meso – Mesocosm temperature and proxy arrays.
proxy_meso – Mesocosm temperature and proxy arrays.
t_crtp – Coretop temperature and proxy arrays.
proxy_crtp – Coretop temperature and proxy arrays.
gdgt23ratio_crtp – GDGT-2/GDGT-3 ratio for coretop samples. Sets use_gdgt23ratio=1 if non-zero/non-NaN.
sd_gdgt23ratio_crtp – Per-site measurement SE of gdgt23ratio (same units, linear). Required for the _eiv model; always included in the data dict (defaults to zeros when not provided, which disables the G₂/₃ EIV measurement model).
no3_crtp – Nitrate concentration for coretop samples. Sets use_no3=1 if non-zero/non-NaN.
sd_no3_crtp – Per-site measurement SE of NO₃ (μmol/L, linear space). Required for the _eiv model. Always included (defaults to zeros; sites with sd=0 receive only the lognormal prior and skip the normal measurement model).
no3_cutoff (float | None) – NO3 threshold for the nonthermal correction. Priority: (1) this arg, (2) culmeso_posterior attrs, (3) auto-calculated via Spearman method.
proxy_residuals_crtp (ndarray | None) – Pre-computed proxy residuals for NO3 threshold calculation. If omitted, residuals are computed internally by fitting a generalized logistic curve. Warning: Stan models use generalized logistic — residuals from other functional forms may shift the threshold.
sd_proxyObs – Per-site analytical SE of the proxy (Scaled RI units). Used by the _eiv model; defaults to 0.03 for every site (Schouten et al. 2013).
R2_thermal (float | None) – R² of a temperature-only coretop fit. Required by the _eiv model, where it scales the process-noise prior; omitted from the dict when not given.
culmeso_posterior (Dataset | None) – xr.Dataset from a completed culmeso forward run. Auto-extracts prior_mean_*/prior_sd_* hyperpriors and no3_cutoff (if saved in attrs).
prior_mean_*/prior_sd_* – Manual hyperprior values. Override auto-extracted values from culmeso_posterior for individual params.
- Returns:
- Stan-ready data dict with proxyObs_* keys, N_* counts, use_* flags,
and hyperpriors — ready for get_posterior().
- Return type:
dict
Build invT input data#
- build_invT_inputData(proxyObs, prior_mu_t, prior_sigma_t, *, fwd_posterior_name=None, predictors=None, config=None, fwd_posterior=None, fwd_cache_dir=None)[source]#
Build the data dictionary for Stan’s inverse model and sampler configuration.
WORKFLOW: ───────── 1. Load forward calibration posterior from .nc file (or accept a pre-loaded Dataset) 2. Randomly sample M parameter sets from that posterior 3. Extract calibration curve parameters (t0, k, b, v, sigma) 4. Package optional environmental predictors (GDGT-2/3, NO3) if used 5. Return data dict (for Stan) + sampler_kwargs (for CmdStanPy)
- Parameters:
proxyObs (ndarray | List[float]) – Observed proxy values to predict temperature from (length N). Any proxy is accepted: scaledRI, TEX86, ringIndex, etc.
prior_mu_t (ndarray | float) – Prior mean temperature (scalar or array of length N)
prior_sigma_t (float) – Prior temperature uncertainty (e.g., 10°C)
fwd_posterior_name (str | None) – Name of saved forward calibration (without .nc extension). Not required when fwd_posterior is supplied directly.
predictors (Dict[str, ndarray] | None) – Optional environmental covariates {‘gdgt23ratio’: array, ‘no3’: array}
config (InvTConfig | None) – Configuration object controlling M, seed, etc.
fwd_posterior (Dataset | None) – Pre-loaded forward posterior xr.Dataset. When provided, fwd_posterior_name is ignored and no file I/O is performed. Useful when running from Google Colab or any pip-install context where the posterior cache is not available.
fwd_cache_dir (str | Path | None) – Directory to resolve fwd_posterior_name in. Defaults to the standard forward posterior cache. Use this to read posteriors from a project-local bundle without copying them into the cache.
- Returns:
Dictionary for Stan’s data block sampler_kwargs: Dictionary for CmdStanPy sampling configuration
- Return type:
data
InvT configuration#
- class InvTConfig(mode='ensemble', n_draws=None, seed=42, no3_cutoff=0.0, suffix=None)[source]#
Knobs for the forward-to-inverse bridge in
build_invT_inputData.Controls how many parameter sets M are drawn from the forward posterior and marginalised over. The marginal-likelihood error falls as 1/sqrt(M) while cost rises linearly with it, so M is the main accuracy-versus-runtime dial of a reconstruction.
- Parameters:
mode (str)
n_draws (int | None)
seed (int)
no3_cutoff (float | None)
suffix (str | None)
- mode#
Must be
"ensemble"(the only supported mode; kept for backward compatibility).- Type:
str
- n_draws#
M, the number of forward draws.
Noneauto-selectsmin(500, max(100, available_draws // 4))– for a typical 4-chain x 1000-draw forward fit (4000 draws) this resolves to 500. The paper used this default (M = 500) for its downcore reconstructions andn_draws=300(~5.8% Monte Carlo error) for its global coretop analyses; pass a value explicitly to trade accuracy for speed.- Type:
int | None
- seed#
Seed for selecting those M draws, so a reconstruction is reproducible.
- Type:
int
- no3_cutoff#
Fallback NO3 cutoff (umol/L), used only when the forward posterior carries no
no3_cutoffattr of its own – the posterior always wins.- Type:
float | None
- suffix#
Force a forward parameter suffix (e.g.
"crtp") instead of taking the highest-priority one present.- Type:
str | None
Modern NO₃ lookup#
- lookup_modern_no3(lat, lon, ocean_dataset, variable='no3_sf2tc_avg', method='linear', *, lat_name=None, lon_name=None)[source]#
Look up modern NO₃ at one or more lat/lon coordinates from the
ocean_prop_dsocean-properties Dataset.The dataset is typically the preprocessed
ocean_prop_dsgenerated in SI_code00 (or fetched byget_ocean_prop_ds()), which contains the thermocline-depth-integrated CMEMS nitrate climatology on the regular WOA23(lat, lon)grid. The returned value(s) are time-invariant (climatological mean) and intended as a modern-ocean proxy for the NO₃ correction in paleo reconstructions.- Parameters:
lat (float or array-like) – Latitude(s) in decimal degrees (−90 to 90). Pass a scalar for a single drill site; pass an array of length N to match N observations.
lon (float or array-like) – Longitude(s) in decimal degrees. Both −180–180 and 0–360 conventions are accepted — the function normalizes to match the dataset’s convention automatically.
ocean_dataset (xr.Dataset) – The
ocean_prop_dsfield (or any dataset) on a regular latitude/longitude grid containing variable. The axis names are detected (lat/lon,latitude/longitude,LAT/LON, …); pass lat_name / lon_name to override.variable (str) – Name of the NO₃ variable to extract. Default
"no3_sf2tc_avg"(thermocline depth-integrated CMEMS annual mean from SI_code00).method ({"linear", "nearest"}) – Interpolation method.
"linear"(default) performs bilinear interpolation and is preferred for smooth fields."nearest"snaps to the closest grid cell and is useful when the dataset is sparse or has NaN-masked shelves.lat_name (str, optional) – Latitude coordinate name. Auto-detected when omitted.
lon_name (str, optional) – Longitude coordinate name. Auto-detected when omitted.
- Returns:
NO₃ value(s) in µmol/L. Shape matches the scalar/array input: a 0-d array for scalar inputs, 1-d array of length N for array inputs. NaN is returned for locations outside the dataset’s valid range (e.g. continental shelves masked in the CMEMS product).
- Return type:
np.ndarray
- Raises:
KeyError – If variable is not found in ocean_dataset.
ValueError – If the latitude/longitude axes cannot be resolved, or the resolved names are coordinates but not dimensions.
Examples
Single drill site:
>>> no3_val = lookup_modern_no3(15.3, -23.7, ocean_prop_ds) >>> # returns scalar-equivalent float; broadcasts to all N obs automatically >>> result = predict_T_from_proxyObs(..., no3=no3_val)
Multi-site stack (per-obs lookup):
>>> no3_arr = lookup_modern_no3(core_df["lat"].values, ... core_df["lon"].values, ... ocean_prop_ds) >>> result = predict_T_from_proxyObs(..., no3=no3_arr)
- lookup_no3_from_woa(lat, lon, woa_dataset, variable='no3_sf2tc_avg', method='linear', *, lat_name=None, lon_name=None)[source]#
Deprecated alias of
lookup_modern_no3().The name was wrong: the default
no3_sf2tc_avgvariable is the CMEMS nitrate climatology, not WOA23 (see the module docstring). Kept so existing calls keep working; emits aDeprecationWarning.- Parameters:
lat (float | ndarray)
lon (float | ndarray)
woa_dataset (Dataset)
variable (str)
method (Literal['linear', 'nearest'])
lat_name (str | None)
lon_name (str | None)
- Return type:
ndarray
Forward calibration#
Get posterior#
- get_posterior(data, stan_file, temptype, proxy_name, *, iter_warmup=None, iter_sampling=None, threads_per_chain=None, chains=None, parallel_chains=None, adapt_delta=None, max_treedepth=None, **kwargs)[source]#
Run a forward calibration in Stan and return the posterior.
Wraps
StanSamplerwith automatic predictor detection, CPU configuration and metadata attachment. The returned dataset can be passed straight topredict_proxy_from_Tor saved withsave_posterior.- Parameters:
data (dict) – Stan data dict from
build_fwd_data(). Theuse_*predictor flags are auto-detected from the arrays present.stan_file (str) – Model name without
.stan, e.g."gen_logi_fixed_hier_crtp_multiv_priorApprox_eiv".temptype (str) – Temperature target, e.g.
"SST"or"thermoT".proxy_name (str) – Proxy label, e.g.
"scaledRI_cren3". Required: it is written to the attrs and validated when the posterior is later used for a reconstruction.iter_warmup (int | None) – Warmup iterations per chain. Default 1000 (CmdStan’s).
iter_sampling (int | None) – Sampling iterations per chain. Default 1000.
threads_per_chain (int | None) – Threads per chain for
reduce_summodels; auto-enabled for models whose filename containsreduce_sum.chains (int | None) – Independent chains. Default 4.
parallel_chains (int | None) – Chains run at once. Auto-detected from the CPU count.
adapt_delta (float | None) – Target acceptance rate. Default 0.8; raise toward 0.99 to trade leapfrog steps for fewer divergences.
max_treedepth (int | None) – HMC maximum tree depth. Default 10.
**kwargs – Forwarded to
CmdStanModel.sample.
- Returns:
an
xr.Datasetof draws with metadata attrs (model name, temptype, proxy_name, priors,stan_diag_*), and the human-readable sampler diagnostic summary.- Return type:
(posterior, diagnostics)- Raises:
ValueError – if active predictors are present but a univariate model was requested, or if an EIV model is requested without
R2_thermal.
Example
>>> posterior, diag = get_posterior( ... data, stan_file="gen_logi_fixed_hier_crtp_univ_priorApprox", ... temptype="SST", proxy_name="scaledRI_cren3") >>> save_posterior(posterior)
Save posterior#
- save_posterior(posterior, cache_dir=None, overwrite=True, filename_suffix='', layout='auto', run='auto')[source]#
Save a forward-model posterior to disk as compressed NetCDF.
The filename is auto-generated from the posterior’s metadata attrs:
{model}_{temptype}[_gdgt23ratio][_no3_{cutoff}][_{proxy_name}]{suffix}.nc- Parameters:
posterior (xr.Dataset) – Forward calibration posterior returned by
get_posterior(). Must havestan_model_name,temptype, andproxy_nameattrs set (proxy_nameis required — a warning is raised if missing).cache_dir (str or Path, optional) – Directory to write the file. Defaults to the standard forward posterior cache (
data/cache/TEXAS_posterior_cache/for source installs,~/.texas/cache/TEXAS_posterior_cache/for pip installs).overwrite (bool) – If
False, raiseFileExistsErrorwhen the output path already exists. DefaultTrue.filename_suffix (str, optional) – Extra tag appended before
.ncunder the legacy layout, and the run/member token under the case layout. Leading/trailing underscores are stripped. Do not pass a date here. Filenames do not carry date stamps; the run date is recorded in therun_timestampattr, where it survives a rename and does not have to be parsed back out of a path. Preferrun=for an explicit member.run (str or int, optional) – Optional run/member token under the case layout.
"auto"(default) writes no member, so a configuration has one canonical path and a re-run replaces it; pass a value only to keep a run deliberately apart. An explicit run wins over filename_suffix for the case path. Withoverwrite=Falsean existing file raisesFileExistsError.layout ({"auto", "case", "legacy"}) – Where to write.
"case"uses the flat CESM-style case name (tx.GHEB.sst.sri03.G23-N1p0.fwd.nc);"legacy"uses the historical long flat filename;"auto"(default) prefers the case layout and falls back to legacy with a warning if no case id can be derived. SeeTEXAS.utils.naming.
- Returns:
Absolute path of the saved
.ncfile.- Return type:
Path
Load posterior#
- load_posterior(model_name, model_type='forward', cache_dir=None)[source]#
Load a posterior from disk: {model_name}.nc in the appropriate cache directory.
- Parameters:
model_name (str) – Name of the model file (without .nc extension)
model_type (Literal['forward', 'invT']) – Type of posterior (“forward” or “invT”)
cache_dir (str | Path | None) – Custom cache directory (overrides default locations)
- Returns:
xarray.Dataset containing the posterior
- Raises:
FileNotFoundError – If the posterior file doesn’t exist
- Return type:
Dataset
Ensemble#
Generate ensemble (auto)#
- generate_ensemble_auto(post_ds, x_vals, model_type='auto', gdgt23ratio=None, no3=None, no3_cutoff=None, return_full_ensemble=False, suffix=None, **kwargs)[source]#
Sample draws from a forward posterior and compute calibration-curve percentiles.
Inspects the posterior’s
stan_model_nameattr anddata_varsto determine the model function, parameter names, and optional-predictor flags automatically, then delegates togenerate_ensemble.- Parameters:
post_ds (xr.Dataset) – Forward calibration posterior returned by
get_posterior()or loaded withload_posterior().x_vals (np.ndarray) – Temperature values (°C) at which to evaluate the calibration curve.
model_type ({"auto", "forward", "inverse"}) – Force forward or inverse dispatch;
"auto"(default) infers from the posterior. InvT posteriors are not supported — usepredict_T_from_proxyObs()instead.gdgt23ratio (np.ndarray, optional) – GDGT-2/3 ratio values; required when the posterior was fitted with a multivariate (GDGT-2/3) model.
no3 (np.ndarray, optional) – NO₃ concentrations (µmol/L); required when the posterior uses the NO₃ correction.
no3_cutoff (float, optional) – Override the NO₃ cutoff from the posterior attrs.
return_full_ensemble (bool) – If
True, return the full M × N draw matrix in addition to percentiles. DefaultFalse.suffix (str, optional) – Force a specific parameter suffix (e.g.
"crtp"); overrides auto-detection.**kwargs – Forwarded to
generate_ensemble.
- Returns:
Keys
"p1"…"p99"(and optionally"ensemble") — each a numpy array of lengthlen(x_vals).- Return type:
dict
- Raises:
NotImplementedError – If called with an invT posterior.
Detect model and params#
- detect_model_and_params(posterior_ds, suffix=None)[source]#
Infer the curve, parameter names and predictor flags from a forward posterior.
Structure is read from the posterior’s
data_varsand attrs, never fromstan_model_name, so a posterior that was renamed or downloaded from Zenodo still dispatches correctly. The presence of avparameter selects the generalized curve over the plain logistic;gamma_G23*/gamma_NO3*select the T0-shift parameterization over the additive one.- Parameters:
posterior_ds (Dataset) – A forward calibration posterior.
suffix (str) – Force a parameter suffix (e.g.
"crtp").Noneselects by the priority order crtp > culmesocore > culmeso > meso > cul.
- Returns:
A dict with
model_function(the callable to evaluate),param_names(unsuffixed names, in the order the callable wants them),suffix,is_multivariate,use_gdgt23ratio,use_no3andno3_cutoff.- Raises:
ValueError – if gamma coefficients are present without a
vparameter. No shipped model produces that combination, and fitting the additive form to gamma coefficients would be wrong by a whole parameterization while still looking plausible.
Diagnostics#
Sampler diagnostics#
- summarize_sampler_diagnostics(fit)[source]#
Summarize a CmdStanPy fit’s convergence diagnostics as
stan_diag_*attrs.Reads divergent transitions, max-treedepth saturation, E-BFMI, R-hat and bulk ESS, and grades each against a fixed threshold, so a run can be accepted or rejected without re-opening the raw fit. Attached to every posterior by
StanSampler.sampleandget_invT_posterior.- Parameters:
fit – A
CmdStanMCMCobject, as returned byCmdStanModel.sample.- Returns:
the counts and percentages (
n_divergent,pct_divergent,n_max_treedepth,pct_max_treedepth,min_ebfmi,max_rhat,n_high_rhat,min_ess_bulk), a*_statusof"PASS","FAIL"or"UNKNOWN"beside each, andstan_diag_overall_status. The thresholds are: divergences < 1%, max-treedepth < 5%, R-hat < 1.01, bulk ESS > 100. All values are plain Python scalars, so the dict can be written straight into NetCDF attrs.- Return type:
A flat dict of
stan_diag_*entries
Summary table#
- create_summary_table(datasets)[source]#
Tabulate the
stan_diag_*attrs of several posteriors, one row each.- Parameters:
datasets (list) – Posteriors carrying
stan_diag_*attrs, as written bysummarize_sampler_diagnostics().- Returns:
A DataFrame with a
modelcolumn (each dataset’sfilenameattr, or"unknown") plus one column per diagnostic, with thestan_diag_prefix stripped. A dataset missing a diagnostic getsNaNin that column rather than being dropped.- Return type:
DataFrame
Plotting#
Plot prior distributions#
- plot_prior_distributions(priors_list=None, posterior_datasets=None, posterior_labels_list=None, show_suptitle=True, kde_bw=0.3, focus_on_posterior=True, include_groups=('t0', 'k', 'b', 'v', 'a', 'beta_G23', 'beta_NO3', 'gamma_G23', 'gamma_NO3', 'betaLogit_G23', 'betaLogit_NO3', 'sigma_proxyObs'), suffix_include=None, zoomin_suffix=None, zoomin_dataset_idx=None, use_linestyle_by_param=False, show_histogram=True, show_annotation=False, set_linewidth=1.5, set_fig_width_factor=3, set_fig_height_factor=3.5, set_leg_max_ncol=3, color_list=None, param_source_map=None, annotation_style='ci95', show_subplot_legend=True, show_figure_legend=True, show_prior_expression=True, cache_dir=None)[source]#
Plot priors + any number of posterior distributions in a grid, split by parameter group (t0, k, b, etc.).
- Parameters:
param_source_map (Dict[str, int] | None) –
Optional dict mapping a param group name to the index of the dataset in
posterior_datasetsthat should be used as the sole source for that group. All other datasets are skipped for that group.Use this when different parameters come from different posteriors — e.g. logistic params (t0, k, b…) from a
culmesorun and beta coefficients from a multivariatecrtprun:plot_prior_distributions( posterior_datasets=[culmeso_ds, crtp_multiv_ds], param_source_map={"beta_G23": 1, "beta_NO3": 1}, )
When a group is not in
param_source_map, all datasets are searched as usual.include_groups (Sequence[str]) – Parameter-group prefixes to draw, in panel order. The default covers the thermal parameters plus every coefficient naming scheme in
PREDICTOR_GROUPS(beta_*for the parent model,gamma_*for t0shift,betaLogit_*for boundedCeil), so the same call works for any of them and for figures mixing several. A group with no matching variable in any dataset is skipped, as is a coefficient group whose predictor no dataset switched on.cache_dir (str | Path | None) – Directory to resolve any name strings in
posterior_datasets. Defaults to the standard forward posterior cache. Loaded Datasets are passed through untouched.priors_list (List[str] | Dict[str, str] | None)
posterior_datasets (List[xr.Dataset] | None)
posterior_labels_list (List[str] | None)
show_suptitle (bool)
kde_bw (float)
focus_on_posterior (bool)
suffix_include (List[str] | None)
zoomin_suffix (str | List[str] | None)
zoomin_dataset_idx (int | None)
use_linestyle_by_param (bool)
show_histogram (bool)
show_annotation (bool)
set_linewidth (float)
set_fig_width_factor (float)
set_fig_height_factor (float)
set_leg_max_ncol (int)
color_list (Sequence[str] | None)
annotation_style (Literal['ci95', 'ci68', 'sigma'])
show_subplot_legend (bool)
show_figure_legend (bool)
show_prior_expression (bool)