8000 Include org details from the manifest into the RO-Crate by muffato · Pull Request #4628 · nf-core/tools · GitHub
[go: up one dir, main page]

Skip to content

Include org details from the manifest into the RO-Crate - #4628

Draft
muffato wants to merge 10 commits into
devfrom
manifest_org_details
Draft

muffato wants to merge 10 commits into
devfrom
manifest_org_details

Conversation

@muffato
@muffato muffato commented Nov 20, 2026 •
Copy link
Copy Markdown
Member

Alternative to #4146

My goal here is to make the RO-Crates more accurate by using actual correct org details (URL and name) for 3rd-party orgs that use the nf-core tooling. RO-Crates are indeed automatically updated by nf-core at every template update and version bump. Fixing them every time is quite time consuming.

This time, rather than introducing template parameters, I hijack existing manifest values:

And simply use them instead of https://nf-co.re and nf-core, while still defaulting to these values when the keys are not present in the manifest.
This way, in usual nf-core pipelines, since these are not used, nothing changes. And external orgs now have the possibility to define them as they wish.

This makes the change much shorter, with far less maintenance burden than my previous attempt.

However, it expects:

  1. The docsUrl URL to have a /pipelines.json that must be formatted the same way the nf-core one is
  2. URLs <docsUrl>/$pipeline_name/$version to be valid

PR checklist

  • This comment contains a description of changes (with reason)
  • CHANGELOG.md is updated
  • If you've fixed a bug or added code that should be tested, add tests!
  • Documentation in docs is updated

@codecov
codecov Bot commented Nov 20, 2026 •
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.91%. Comparing base (41500d7) to head (d35a60b).
⚠️ Report is 1 commits behind head on dev.

Files with missing lines Patch % Lines
nf_core/pipelines/rocrate.py 87.50% 2 Missing ⚠️

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codspeed
codspeed Bot commented Nov 20, 2026 •
Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

✅ 9 untouched benchmarks


Comparing manifest_org_details (d35a60b) with dev (41500d7)1

Open in CodSpeed

Footnotes

  1. No successful run was found on dev (cd7c955) during the generation of this report, so 41500d7 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@muffato muffato self-assigned this Nov 20, 2026
@muffato
muffato force-pushed the manifest_org_details branch from 98ad5ea to 4317734 Compare November 20, 2026 15:06
@muffato
muffato marked this pull request as ready for review November 20, 2026 15:06
@prototaxites
Copy link
Copy Markdown
Contributor

docsUrl

Is this something where we should actually request an organizationUrl in the manifest? I would rather expect this manifest parameter to point directly at the docs.

self.crate.mainEntity.append_to(
"url", f"https://nf-co.re/{self.crate.name.replace('nf-core/', '')}/{url}/", compact=True
)
self.crate.mainEntity.append_to("url", f"{org_website}/{pipeline_short_name}/{url}/", compact=True)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we happy to assume that non-nf-core pipeline webistes use this structure and have a pipelines.json file?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converting back to draft because I've got an idea to tackle this

@muffato
muffato commented Nov 20, 2026
Copy link
Copy Markdown
Member Author

Are we happy to assume that non-nf-core pipeline webistes use this structure and have a pipelines.json file?

By the way there's a fallback in the code.

Is this something where we should actually request an organizationUrl in the manifest? I would rather expect this manifest parameter to point directly at the docs.

Those are the main questions indeed. There are two URL fields in the manifest: homePage is already set to the GitHub URL, so docsUrl is the only one I had left.
I'd argue that https://nf-co.re is where all the docs are, so it can work as docsUrl 😅

@eduard-watchmaker eduard-watchmaker left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One regression and a few small things:

  1. rocrate.py:194 and :218: the template doesn't set docsUrl, so nf-core pipelines use the default. .rstrip("/") turns it into https://nf-co.re, so the Organization @id/url and sdPublisher change from https://nf-co.re/. Every pipeline's ro-crate-metadata.json would change at the next sync. Could the default stay as is, and the slash only be stripped when building paths?
  2. manifest.get(key, default) only falls back when the key is missing. docsUrl = '' or organization = '' gives an empty @id. Using manifest.get("docsUrl") or "https://nf-co.re/" and computing it once would cover both.
  3. rocrate.py:239: topics = ["nf-core", "nextflow"] still tags every crate with nf-core. org_name would fit the goal of the PR.
  4. The new test checks only the Organization name and url. Assertions on sdPublisher, mainEntity.url, the topics fetched from the mocked pipelines.json, and the default case would catch point 1.

This was generated assisted by Claude code and checked by me.

@muffato
muffato force-pushed the manifest_org_details branch from 9bbca77 to f5005aa Compare November 20, 2026 22:49
@muffato
muffato force-pushed the manifest_org_details branch from 165fba8 to d35a60b Compare November 20, 2026 23:06
@muffato
muffato commented Nov 20, 2026
Copy link
Copy Markdown
Member Author

Thanks @eduard-watchmaker !

  1. Not important because the ro-crates aren't stable in the first place. They contain timestamps and UUIDs that change all the time
  2. Fixed
  3. I had missed that. Fixed
  4. Tests updated

@eduard-watchmaker eduard-watchmaker left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, all points are addressed. Fair point on the trailing slash, as the crates change on every sync anyway.

This was generated assisted by Claude code and checked by me.

@muffato
muffato marked this pull request as draft November 21, 2026 13:36

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

0