Repository rules to fetch third-party npm packages
These use Bazel's downloader to fetch the packages. You can use this to redirect all fetches through a store like Artifactory.
See https://blog.aspect.dev/configuring-bazels-downloader for more info about how it works and how to configure it.
translate_pnpm_lock
is the primary user-facing API.
It uses the lockfile format from pnpm because it gives us reliable
semantics for how to dynamically lay out node_modules
trees on disk in bazel-out.
To create pnpm-lock.yaml
, consider using pnpm import
to preserve the versions pinned by your existing package-lock.json
or yarn.lock
file.
If you don't have an existing lock file, you can run npx pnpm install --lockfile-only
.
Advanced users may want to directly fetch a package from npm rather than start from a lockfile.
npm_import
does this.
Rules
npm_import
Import a single npm package into Bazel.
Normally you'd want to use translate_pnpm_lock
to import all your packages at once.
It generates npm_import
rules.
You can create these manually if you want to have exact control.
Bazel will only fetch the given package from an external registry if the package is required for the user-requested targets to be build/tested.
This is a repository rule, which should be called from your WORKSPACE
file
or some .bzl
file loaded from it. For example, with this code in WORKSPACE
:
npm_import(
name = "npm__at_types_node_15.12.2",
package = "@types/node",
version = "15.12.2",
integrity = "sha512-zjQ69G564OCIWIOHSXyQEEDpdpGl+G348RAKY0XXy9Z5kU9Vzv1GMNnkar/ZJ8dzXB3COzD9Mo9NtRZ4xfgUww==",
)
This is similar to Bazel rules in other ecosystems named "_import" like
apple_bundle_import
,scala_import
,java_import
, andpy_import
.go_repository
is also a model for this rule.
The name of this repository should contain the version number, so that multiple versions of the same package don't collide. (Note that the npm ecosystem always supports multiple versions of a library depending on where it is required, unlike other languages like Go or Python.)
To consume the downloaded package in rules, it must be "linked" into the link package in the
package's BUILD.bazel
file:
load("@npm__at_types_node_15.12.2//:node_package.bzl", node_package_types_node = "node_package")
node_package_types_node()
This instantiates a node_package
target for this package that can be referenced by the alias
@//link/package:npm__name
and @//link/package:npm__@scope+name
for scoped packages.
The npm
prefix of these alias is configurable via the namespace
attribute.
When using translate_pnpm_lock
, you can link
all the npm dependencies in the lock file with:
load("@npm//:node_modules.bzl", "node_modules")
node_modules()
translate_pnpm_lock
also creates convienence aliases in the external repository that reference
the linked node_package
targets. For example, @npm//name
and @npm//@scope/name
.
To change the proxy URL we use to fetch, configure the Bazel downloader:
Make a file containing a rewrite rule like
rewrite (registry.nodejs.org)/(.*) artifactory.build.internal.net/artifactory/$1/$2
To understand the rewrites, see UrlRewriterConfig in Bazel sources.
Point bazel to the config with a line in .bazelrc like common --experimental_downloader_config=.bazel_downloader_config
Example usage (generated)
load("@aspect_rules_js//js:npm_import.bzl", "npm_import")
npm_import(
# A unique name for this repository.
name = "",
# Name of the npm package, such as `acorn` or `@types/node`
package = "",
# A dictionary from local repository name to global repository name
repo_mapping = {},
# Version of the npm package, such as `8.4.0`
version = "",
)
name
A unique name for this repository.
deps
A dict other npm packages this one depends on where the key is the package name and value is the version
enable_lifecycle_hooks
If true, runs lifecycle hooks declared in this package and the custom postinstall script if one exists.
indirect
If True, this is a indirect npm dependency which will not be linked as a top-level node_module.
integrity
Expected checksum of the file downloaded, in Subresource Integrity format. This must match the checksum of the file downloaded.
This is the same as appears in the pnpm-lock.yaml, yarn.lock or package-lock.json file.
It is a security risk to omit the checksum as remote files can change. At best omitting this field will make your build non-hermetic. It is optional to make development easier but should be set before shipping.
link_package_guard
When explictly set, check that the generated node_package() marcro in package.bzl is called within the specified package.
Default value of "." implies no gaurd.
This is set by automatically when using translate_pnpm_lock via npm_import to guard against linking the generated node_modules into the wrong location.
package
Name of the npm package, such as acorn
or @types/node
patch_args
Arguments to pass to the patch tool.
-p1
will usually be needed for patches generated by git.
patches
Patch files to apply onto the downloaded npm package.
postinstall
Custom string postinstall script to run against the installed npm package. Runs after any existing lifecycle hooks.
repo_mapping
A dictionary from local repository name to global repository name. This allows controls over workspace dependency resolution for dependencies of this repository.
For example, an entry "@foo": "@bar"
declares that, for any time this repository depends on @foo
(such as a dependency on @foo//some:target
, it should actually resolve that dependency within globally-declared @bar
(@bar//some:target
).
transitive_closure
A dict all npm packages this one depends on directly or transitively where the key is the package name and value is a list of version(s) depended on in the closure.
version
Version of the npm package, such as 8.4.0
yq_repository
The basename for the yq toolchain repository from @aspect_bazel_lib.
translate_pnpm_lock
Repository rule to generate npm_import rules from pnpm lock file.
The pnpm lockfile format includes all the information needed to define npm_import rules, including the integrity hash, as calculated by the package manager.
For more details see, https://github.com/pnpm/pnpm/blob/main/packages/lockfile-types/src/index.ts.
Instead of manually declaring the npm_imports
, this helper generates an external repository
containing a helper starlark module repositories.bzl
, which supplies a loadable macro
npm_repositories
. This macro creates an npm_import
for each package.
The generated repository also contains BUILD files declaring targets for the packages
listed as dependencies
or devDependencies
in package.json
, so you can declare
dependencies on those packages without having to repeat version information.
Bazel will only fetch the packages which are required for the requested targets to be analyzed. Thus it is performant to convert a very large pnpm-lock.yaml file without concern for users needing to fetch many unnecessary packages.
Setup
In WORKSPACE
, call the repository rule pointing to your pnpm-lock.yaml file:
load("@aspect_rules_js//js:npm_import.bzl", "translate_pnpm_lock")
# Read the pnpm-lock.yaml file to automate creation of remaining npm_import rules
translate_pnpm_lock(
# Creates a new repository named "@npm_deps"
name = "npm_deps",
pnpm_lock = "//:pnpm-lock.yaml",
)
Next, there are two choices, either load from the generated repo or check in the generated file. The tradeoffs are similar to this rules_python thread.
- Immediately load from the generated
repositories.bzl
file inWORKSPACE
. This is similar to thepip_parse
rule in rules_python for example. It has the advantage of also creating aliases for simpler dependencies that don't require spelling out the version of the packages. However it causes Bazel to eagerly evaluate thetranslate_pnpm_lock
rule for every build, even if the user didn't ask for anything JavaScript-related.
load("@npm_deps//:repositories.bzl", "npm_repositories")
npm_repositories()
In BUILD files, declare dependencies on the packages using the same external repository.
Following the same example, this might look like:
js_test(
name = "test_test",
data = ["@npm_deps//@types/node"],
entry_point = "test.js",
)
- Check in the
repositories.bzl
file to version control, and load that instead. This makes it easier to ship a ruleset that has its own npm dependencies, as users don't have to install those dependencies. It also avoids eager-evaluation oftranslate_pnpm_lock
for builds that don't need it. This is similar to theupdate-repos
approach from bazel-gazelle.
In a BUILD file, use a rule like write_source_files to copy the generated file to the repo and test that it stays updated:
write_source_files(
name = "update_repos",
files = {
"repositories.bzl": "@npm_deps//:repositories.bzl",
},
)
Then in WORKSPACE
, load from that checked-in copy or instruct your users to do so.
In this case, the aliases are not created, so you get only the npm_import
behavior
and must depend on packages with their versioned label like @npm__types_node-15.12.2
.
name
A unique name for this repository.
dev
If true, only install devDependencies
enable_lifecycle_hooks
If true, runs lifecycle hooks on installed packages as well as any custom postinstall scripts
no_optional
If true, optionalDependencies are not installed
node_repository
The basename for the node toolchain repository from @build_bazel_rules_nodejs.
package
The package to "link" the generated npm dependencies to. By default, the package of the pnpm_lock target is used.
patch_args
A map of package names or package names with their version (e.g., "my-package" or "my-package@v1.2.3") to a label list arguments to pass to the patch tool. Defaults to -p0, but -p1 will usually be needed for patches generated by git. If patch args exists for a package as well as a package version, then the version-specific args will be appended to the args for the package.
patches
A map of package names or package names with their version (e.g., "my-package" or "my-package@v1.2.3")
to a label list of patches to apply to the downloaded npm package. Paths in the patch
file must start with extract_tmp/package
where package
is the top-level folder in
the archive on npm. If the version is left out of the package name, the patch will be
applied to every version of the npm package.
pnpm_lock
The pnpm-lock.yaml file.
postinstall
A map of package names or package names with their version (e.g., "my-package" or "my-package@v1.2.3")
to a string postinstall script to apply to the downloaded npm package after its existing postinstall script runs.
If the version is left out of the package name, the script will run on every version of the npm package. If
postinstall scripts exists for a package as well as for a specific version, the script for the versioned package
will be appended with &&
to the non-versioned package script.
prod
If true, only install dependencies
repo_mapping
A dictionary from local repository name to global repository name. This allows controls over workspace dependency resolution for dependencies of this repository.
For example, an entry "@foo": "@bar"
declares that, for any time this repository depends on @foo
(such as a dependency on @foo//some:target
, it should actually resolve that dependency within globally-declared @bar
(@bar//some:target
).
yq_repository
The basename for the yq toolchain repository from @aspect_bazel_lib.