8000 extern: improves the handling of external dependencies in a conda environment by aivazis · Pull Request #38 · aivazis/mm · GitHub
[go: up one dir, main page]

Skip to content

extern: improves the handling of external dependencies in a conda environment - #38

Merged
aivazis merged 8 commits into
mainfrom
conda-pkgdb
Jun 6, 2026
Merged

aivazis merged 8 commits into
mainfrom
conda-pkgdb

Conversation

@aivazis
@aivazis aivazis commented Jun 6, 2026
Copy link
Copy Markdown
Owner

Completes the conda package-database rework: guarantees every supported external is accounted for, lets mm verify that a configured package can actually be compiled and linked against, and fixes two latent config bugs found along the way.

Built as five focused commits.

a) Recipe coverage + exception registry (d0c881f)

Every package under make/extern/ must now be either a _condaRecipes entry or a registered _condaExceptions entry (with a reason); a stray directory triggers a journal.warning at --pkgdb=conda --setup (warn, never block). Exceptions: fmm3d/libtorch (user-managed), summit (source-only), p2 (pyre-next sandbox), fortran (toolchain pseudo-package, contributes the compiler's fortran runtime at link time). pyre is now a canonical recipe — it is on conda-forge — but absence from a conda env is a silent omission, never a disable, so source installs are unaffected.

b) Per-package header markers (83c88d6)

Each package declares <pkg>.markers.headers — non-exhaustive header sentinels whose absence on incpath proves breakage. A marker is the #include string relative to incpath, so the one field handles both layouts: civilized packages carry the prefix (gsl/gsl_version.h), flat ones use the bare name (SpiceUsr.h). Link markers already existed as <pkg>.libraries.

b.1) cspice works out of the box on conda (2adef99)

CSPICE includes its headers flat (<SpiceUsr.h>), but conda relocates them into include/cspice/. New _emitCondaCspice handler reads the install manifest and emits cspice.incpath ?= $(conda.prefix)/include/cspice (collapses to include for a flat source install). Source/macports/ubuntu refinements deferred as later bugs.

yaml → yaml-cpp rename (15a25f8)

make/extern/yaml was actually yaml-cpp squatting on the canonical C libyaml name (it caused a false-positive conda match against libyaml). Renamed the dir + variable prefix, dropped the vestigial flavor hack, fixed the recipe key in all three package managers (conda/macports/dpkg) and cantera's dependency. Frees yaml for a future real libyaml external (TODO).

c) Marker verification (2a3d4d8)

mm extern.<pkg>.verify (full report) and mm extern.verify (verified/broken lists, like extern.info). It probes ${wildcard incpath/marker} for headers and globs lib<name>.{so,dylib,a} over libpath for libraries. Crucially it reads the resolved make variables, so it honors anything overridden in the adhoc ~/.config/mm config — which a package-database-time check cannot see. Scoped to the project's loaded externs, consistent with the existing per-package info targets.

Verification

  • Coverage exact on a live env: 36 dirs = 31 recipes + 5 exceptions, 0 orphans; warning fires on a stray dir.
  • pyre (source-installed) verifies clean via its resolved override path; a forced bad incpath correctly reports unresolved headers.
  • cspice emits include/cspice and its bare marker resolves there.
  • pkgdb builds clean; extern.info/extern.verify work.

Follow-ups (not in this PR)

  • Add a real libyaml yaml external.
  • make/extern/fmm3d/info.mm → rules.mm (stale pre-rename leftover, never loaded).
  • petsc/slepc/openblas markers chosen but not yet verified on a live install (treat deviations as bugs).
  • Mirror the pyre/externals discovery philosophy back into the pyre framework.

…ed exception (adds the exception registry + coverage warning; pyre is now a canonical recipe)
…squatting on the canonical C libyaml name (also fixes the macports/dpkg recipe keys and cantera dependency), freeing yaml for a future libyaml external
…s) as sentinels whose absence on incpath proves breakage; civilized packages carry the <pkg>/ prefix, flat ones use the bare filename
….h> include resolves against the include/cspice subdir conda relocates it into (works out of the box on conda; source/macports/ubuntu treated as later bugs)
…rify) that probes the resolved incpath/libpath, so it honors adhoc overrides a package-database check cannot see
@aivazis aivazis changed the title extern: conda recipe coverage, marker verification, yaml-cpp rename, cspice conda fix extern: improves the handling of external dependencies in a conda environment Jun 6, 2026
aivazis added 3 commits June 6, 2026 11:38
…tale pre-rename leftover; extern.load only includes init.mm + rules.mm), and drop the stray summit comment/debug lines
…om yaml-cpp; conda/macports/dpkg recipes + yaml.h marker; self-add uses filter not findstring so it does not collide with the yaml-cpp substring
…xtern-pile membership, eliminating name collisions like metis/parmetis and yaml/yaml-cpp; covers all 36 self-add lines plus extern.is.supported/available (mpi flavor checks left as-is, no overlap)
@aivazis
aivazis merged commit cb4d767 into main Jun 6, 2026
@aivazis
aivazis deleted the conda-pkgdb branch June 6, 2026 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0