Improve --install

--install option is an interesting feature. When you maintain a big project, it could significantly help you - for example, avoid loading to CI unnecessary heavy dependencies that are used only in some scripts. However, now this option has some significant issues.

At this moment, this option install all dependencies of the script - even if this dependency is already installed at the root of the project.
It's problematic to control versions of dependencies if they are defined as comments of imports - this dependency can be used in some different scripts, the actual tools for updates checking can't get access to versions from those comments.
Dependencies from the monorepo (workspaces) also installed and resolved incorrectly.
It's problematic to configure installing process.

The first issue can be solved by checking the existence of the dependency in the top-level node_modules of the project before installing.
The second can be solved by adding a way to define those versions in a package.json (top-level of the project or defined in another place) without installing those dependencies. In the case of the top-level package.json, it can be optionalDependencies.

At this moment, this option install all dependencies of the script - even if this dependency is already installed at the root of the project.

We can avoid this side effect by using the package manager: npx -p dep1@v1 -p dep2@* -p depN@^3.0.1 -p zx@7.1.0 zx script.js

We can avoid this side effect by using the package manager: npx -p dep1@v1 -p dep2@* -p depN@^3.0.1 -p zx@7.1.0 zx script.js

However, this way is not so convenient as just zx -i and controlling dependencies versions turns to hell.

Today I played with it and wrote a simple zxi wrapper for zx and install helper, 2 additional commits show how they can be used.

They get required versions of packages from lazyDependencies field of package.json, if the required version of the package is already present - it will not be installed, if all of them - npm i will not be called at all. Sure, it works only with the root package.json and node_modules, for zx it could be good also install dependencies of local dependencies - but it can be simply extended for that.

It could be good to have similar functionality just with zx -i.

At this moment, this option install all dependencies of the script - even if this dependency is already installed at the root of the project.

But is it a problem? Will npm check what package is already installed and it will be noop.

It's problematic to control versions of dependencies if they are defined as comments of imports - this dependency can be used in some different scripts, the actual tools for updates checking can't get access to versions from those comments.

I this this is a feature! Each script can have each own versions. One is using ^2, other ^1. Yes ncu will not detect version/packages. For outdated versions we can create separate package for checking and updating comments (zx-check-updates).

Dependencies from the monorepo (workspaces) also installed and resolved incorrectly.

I'm not familiar with this setup. ¯\_(ツ)_/¯

It's problematic to configure installing process.

For example?

The second can be solved by adding a way to define those versions in a package.json (top-level of the project or defined in another place) without installing those dependencies. In the case of the top-level package.json, it can be optionalDependencies.

What about different version per script? Also Keeping deps and version in one place is better.

But is it a problem? Will npm check what package is already installed and it will be noop.

In some cases, this check can take a long time - 2 days ago in one project it took about 5 minutes without installing anything - and it was one of the reasons why I wrote this issue.

I this this is a feature! Each script can have each own versions. One is using ^2, other ^1. Yes ncu will not detect version/packages. For outdated versions we can create separate package for checking and updating comments (zx-check-updates).

A good practice is defining dependencies in one place - each new place where dependencies are defined, even with a new tool for controlling versions - is a pain. It's a good practice to reuse one dependency in different places of the project - however, if required to use different versions, it's not a big problem to specify it as a comment / in a local, not the root, package.json.

I'm not familiar with this setup. ¯_(ツ)_/¯

However, it's a significant use case. For example, here we use core-js-compat from the monorepo, but zx -i will install it from NPM.

For example?

I think that already shown cases are enougth.

What about different version per script?

See above.

In some cases, this check can take a long time - 2 days ago in one project it took about 5 minutes without installing anything - and it was one of the reasons why I wrote this issue.

I guess we can check what node_modules/foo exists, but what about checking the version? Why npm is so slow about it?

A good practice is defining dependencies in one place - each new place where dependencies are defined, even with a new tool for controlling versions - is a pain. It's a good practice to reuse one dependency in different places of the project - however, if required to use different versions, it's not a big problem to specify it as a comment / in a local, not the root, package.json.

We already have package.json and zx scripts can use project deps. The --install feature is useful for standalone scripts, or scripts fetched from the web.

However, it's a significant use case.

Let's explore the problem space )

I guess we can check what node_modules/foo exists, but what about checking the version? Why npm is so slow about it?

At least, NPM uses ranges and checks for new versions.

We already have package.json and zx scripts can use project deps. The --install feature is useful for standalone scripts, or scripts fetched from the web.

Such scripts can be in the scope of the project - for example, for CI is useful granular loading of dependencies - in one task linting, in another NodeJS testing (with different versions, so cache does not work), in one more - browsers, etc. Some scripts can be standalone and used only locally - but too often they should reuse the same dependencies as the rest project.

Let's explore the problem space )

I already updated this comment:

For example, here we use core-js-compat from the monorepo, but zx -i will install it from NPM.

(Note that I forgot that npm i --no-save pkg from npm@7 unconditionally prunes dependencies and that makes this task a little harder - workspaces can be simpler.)

Let’s try to break down to a few smaller parts what easier to implement.

It can be related oven-sh/bun#1459

Is this completed? I don't see anything about it.

I think zx should not try to replace the package manager. I suggest using script composition for complex corner cases instead:

zx -e 'const deps = fs.readJsonSync("package.json") ...' && zx script.js

import {$, cd} from 'zx'

// read your deps from anywhere
const deps = Object.entries(fs.readJsonSync("package.json").dependencies).reduce(...).join(' ')

// need isolated scope?
// cd('your/temp')
// check `./node_modules`, verify ranges with semver, modify deps, etc

// use any package manager for setup
await $`npm i ${deps}`

// use zx to run zx :-)
await $`zx script.js`

@antonmedv

@antonmedv it's not about a replacement of the package manager, it's about calling a package manager when it's required. Just like in this wrapper, but from the box - since for my use cases that's required too frequently.

I revisited the current "--install" behavior and the concerns raised here, and I think the key is to make dependency resolution deterministic before changing the installation behavior.

A possible implementation path would be:

Resolve before installing: check whether an imported package can already be satisfied by the project’s dependency graph (including the top-level "node_modules") before invoking an installer.
Keep versions declarative: allow versions/ranges to be declared in "package.json" (e.g. "optionalDependencies" or another explicitly supported section), while keeping import comments focused on script-level metadata rather than becoming a second dependency manifest.
Handle workspaces explicitly: resolve workspace dependencies according to the workspace/package boundary instead of treating every script as an isolated dependency set.
Make installation policy configurable: provide a clear mechanism for projects to control the package manager, resolution scope, and installation location rather than having "--install" make those decisions implicitly.
Preserve backwards compatibility: existing "--install" usage should continue to work when no project-level dependency metadata is present.

I’d also suggest covering this with integration fixtures for at least:

dependency already available at the project root;
dependency missing and requiring installation;
version/range declared in "package.json";
dependency provided by a workspace package;
conflicting root/workspace versions;
script containing an inline dependency version.

That would give us a well-defined resolution contract before implementation and, importantly, prevent regressions in monorepo setups.

If maintainers are open to this direction, I’d be happy to prototype the resolver + test matrix in a focused PR.

Navigation Menu

Repository navigation

Improve `--install` #525

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Improve --install #525

Description

Activity

antongolub commented on Oct 10, 2022

zloirock commented on Oct 10, 2022

zloirock commented on Oct 10, 2022

antonmedv commented on Oct 10, 2022

zloirock commented on Oct 10, 2022

antonmedv commented on Oct 10, 2022

zloirock commented on Oct 10, 2022

zloirock commented on Oct 15, 2022

antonmedv commented on Oct 15, 2022

zloirock commented on Nov 5, 2022

zloirock commented on Mar 29, 2024

antongolub commented on Mar 29, 2024

zloirock commented on Mar 29, 2024

AryanXCode646 commented on Sep 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Improve `--install` #525