Wiki

• FrontPage
• RecentChanges
• FindPage
• HelpContents
• Javascript/Tutorial

Little tutorial for JS-Team beginners.

The tools

To package node/javascript modules, you need at least:

• devscripts: scripts to make the life of a Debian Package maintainer easier

• pkg-js-tools: collection of tools to aid packaging Node modules in Debian

• git-buildpackage: suite to help with Debian packages in Git repositories (or git-dpm)

• npm2deb: tool to help debianize Node.js modules (version ≥ 0.3.0 is recommended)

• schroot: securely enter a chroot environment

• lintian: static analysis tool for Debian packages

• duck: the Debian URL checker

Run the following command inside the chroot to install the required packages:

$ sudo apt install devscripts pkg-js-tools git-buildpackage npm2deb lintian duck

Little howtos

Little schroot/sbuild howto

1. To build a minimal Debian unstable environment, you can use this:

$ sudo sbuild-createchroot unstable /srv/chroot/unstable-amd64-sbuild
# or to use ccache and eatmydata:
$ sudo sbuild-createchroot --include=eatmydata,ccache unstable /srv/chroot/unstable-amd64-sbuild

2. To update it:

$ sudo sbuild-update unstable-amd64-sbuild -u -g -a -r

3. List built schroot

$ sudo schroot --list

Schroot improvements

1. Save bandwith using apt-cacher-ng

2. Use tmpfs to increase build speed

Little salsa howto

You need to generate a Personal Access Token in Salsa settings. Select all checkboxes in section Select scopes. Copy the tokens value into ~/.devscripts:

SALSA_TOKEN=xxxxxxxxxxx

Verify that it works:

$ salsa whoami
Id      : 1234
Username: myaccount-guest
Name    : My Name
Email   : my@mail.org
State   : active

You must also be member of js-team group. Verify this:

$ salsa list_groups
Id       : 2099
Name     : Debian JavaScript Maintainers
Full path: js-team

You must also set your public OpenPGP key in your Salsa account.

Packaging walkthrough

Working with existing packages

1. Clone locally the package:

$ salsa --group js-team co node-temporary
$ cd node-temporary

Of course you need to configure salsa(1) with a valid GitLab token.

2. Verify debian/gbp.conf to not import upstream .gitignore. It may contain:

   1 [import-orig]
   2 filter = [ '.gitignore', '.git/*', '.git', '.travis.yml' ]
   3 
   4 [import-dsc]
   5 filter = [ '.gitignore', '.git/*', '.git', '.travis.yml' ]

3. Verify that debian/watch is compliant with npmjs registry:

$ debcheck-node-repo && echo OK

If result is bad, investigate manually. If upstream forgot to insert tags in its repository, please open an issue: it is better to use source repository than npm registry (which may contain some generated files)

4. Update the package:

$ uscan
...
$ gbp import-orig --pristine-tar ../node-temporary_9.9.9.orig.tar.xz

5. Fix debian/* files, then push the result:

$ git push origin master upstream pristine-tar
$ git push --tags

Switch test and install to pkg-js-tools

pkg-js-tools provides hooks for auto-configuration, auto-test and auto-install and also autopkgtest.

To enable it:

• debian/control:

  • add Testsuite: autopkgtest-pkg-nodejs header

  • add dh-sequence-nodejs in build dependencies

  • remove debian/install or adapt it if your source package provides more than one binary package (see pkg-js-tools doc)

  • move build commands from debian/rules (override_dh_auto_build) to debian/nodejs/build (especially if you want to build components using debian/nodejs/<component>/build)

  • move test from debian/rules (override_dh_auto_test) to debian/tests/pkg-js/test

  • remove useless files in debian/tests (control, require,...)

Write good tests

Using pkg-js-tools, you just have to examine package.json (scripts -> test) to find upstream test and write it in debian/tests/pkg-js/test. Example:

$ cat debian/tests/pkg-js/test
mocha -R spec --timeout 10000 test/

See pkg-js-tools full example for more.

If debian/tests/pkg-js/test does not exist, pkg-js-tools will automatically add a minimal test (node require(".")).

Don't forget to remove debian/tests/control and debian/tests/require if exists.

Notes on `jest`

jest is a powerful test framework. When using it in Debian packaging test, be careful to

• use dh-sequence-nodejs/pkg-js-tools ≥ 50 (this installs build-files in autopkgtest test tree)

• use jest --ci to avoid writing snapshots

• use jest --ci -u when upstream includes test snapshots in its source tree. This will avoid false-positive autopkgtest regressions if build dependencies are upgraded

• If you have embedded components and both uses jest, avoid conflicts by limiting the main jest using a regexp argument like jest --ci test/*.js.

• you'll often need a debian/tests/pkg-js/files to fix missing files in autopkgtest (missing src/). Don't forget then to include all needed test files. Common example:

babel.config.js
src
test

Notes on `ava`

ava is not available in Debian, but there are some workarounds:

• for simple test files, use node-tape

• for more complex files, use jest (via jest-codemods)

Ava to tape

##   ORIGINAL TEST     ###        TAPE TEST            ##
                        #
import test from 'ava'  #  const test = require('tape')
import foo from '.'     #  const foo = require('.')
                        #
test('bar', t => {      #  test('bar', t=> {
  t.is( foo(0), 0)      #    t.is( foo(0), 0)
}                       #    t.end()
                        #  }

Just to transform in commonjs and add a t.end() at each test; and adapt some function names. For example, t.notThrows becomes t.doesNotThrow. See /usr/share/doc/node-tape/readme.markdown.gz to find them.

Ava to jest

• Launch jest-codemods:

$ npx jest-codemods
? Which parser do you want to use? Babel
? Which test library would you like to migrate from? AVA
? Are you using the global object for assertions? No
? Will you be using Jest on Node.js as your test runner? Yes
? On which files or directory should the codemods be applied? test.js
Executing command: jscodeshift -t [...] test.js --ignore-pattern node_modules --parser babel
Processing 1 files...
Spawning 1 workers...
Sending 1 files to free worker...
All done.
Results:
0 errors
0 unmodified
0 skipped
1 ok

• Test the result:

$ jest --ci --testRegex test.js

It may need a babel file. If so, add a babel.config.json:

{
  "presets": [ "@babel/preset-env" ],
  "plugins": [ "@babel/plugin-transform-runtime" ]
}

Anyway, jest has not exactly the same features than ava. Especially some throwsAsync / notThrowsAsync test should be removed or modified.

Fully test your package

Dependencies test

Launch test in a minimal and up-to-date unstable environment to verify that all needed packages are declared in debian/control:

sbuild -j5 --no-apt-update -d unstable --no-clean-source --run-lintian --lintian-opts='--color always --display-info --display-experimental --pedantic'

or using gbp:

$ gbp buildpackage --git-ignore-branch --git-builder="sbuild -j5 --no-apt-update -d unstable --no-clean-source --run-lintian --lintian-opts='--color always --display-info --display-experimental --pedantic'" --git-export=WC

This also launches lintian. Check the result and fix all errors/warnings (or override them only if this is a false positive) and take care of other lintian checks.

Result test

pkg-js-tools helps to write good autopkgtest. If you write them yourself, remember that you must test installed files, not source ones. Launch autopkgtest (example with schroot - where ../foo.changes is the package being tested):

$ autopkgtest -B ../foo.changes -- schroot unstable-amd64-sbuild

Notes:

In the event when an error like this is  (autopkgtest-schroot requires ephemeral schroot sessions. Set a "union-type" or use a tarball schroot) is thrown after running this command. Edit with your favorite text-editor  /etc/schroot/chroot.d/unstable-amd64-sbuild-<randomly-generated>  file by adding this options to fix it:

union-type=overlay
union-overlay-directory=/dev/shm

and check result. By default, pkg-js-autopkgtest copies only test* files. If some other are needed, creates debian/tests/pkg-js/files and list all needed test files. Example:

$ cat debian/tests/pkg-js/files
Makefile
test.js

If the test depends on a new package, where ../packages/bar.deb is the test dependency, pass the local .deb dependency as follows:

$ autopkgtest -B  '../packages/bar.deb' '../foo.deb' -- schroot unstable-amd64-sbuild

You can also run autopkgtest with sbuild by adding the following lines to ~/.sbuildrc :

   1 $run_autopkgtest = 1;
   2 $autopkgtest_root_args = '';
   3 $autopkgtest_opts = [ '--', 'schroot', '%r-%a-sbuild' ];

URLs test

Simply launch duck in the Debian source directory and check the result.

Clean test

Launch 2 times dpkg-buildpackage -us -uc. If build files are not cleaned, the second will fail. If so, set them in a debian/clean. Example:

$ cat debian/clean
build/
generated.min.js

Starting a new package

First you may check if needed dependencies exists. Example to build ldapjs:

$ pkgjs-depends ldapjs
# ldapjs@2.3.1
DEPENDENCIES:
  node-asn1 (asn1)
  node-assert-plus (assert-plus)
  node-once (once)
  node-vasync (vasync)
  node-verror (verror)

MISSING:
ldapjs
 └── abstract-logging (2.0.1)
 └── backoff (2.5.0)
     └── precond (0.2.3)
 └── ldap-filter (0.3.3)

Here, we can see that 4 modules are missing in Debian to package it: abstract-logging, backoff, ldap-filter and precond. A quick analysis shows that at least ldap-filter could be embedded.

This package can be built in Debian. Let's do it:

$ npm2deb create ldapjs
...

• Observe the list of FIX_ME lines printed at the end of the output.
• Save these lines to a temporary file since npm2deb will not generate them again.
• Fix all the FIX_MEs in the debian directory.

• Once you are done with your changes, run debuild to refresh the generated files

• Run lintian against the newly generated changes file and make sure that there are no errors.

When all is OK and package builds well and lintian is clean, create a git repo:

$ cd node-vasync-2.2.0
$ git init
$ gbp import-dsc ../path/to/node-vasync_2.2.0-1.dsc

Then, you can push it using salsa(1) in the root of Debian source tree.

$ salsa --group js-team --tagpending --desc push_repo .

If you want to push in your personal area, simply launch:

$ salsa --desc push_repo .

Debugging tips

Here're some useful things that will help you debug various things.

• Running node -e "console.log('Hello')" will execute console.log('Hello') in node.

• Running node -e "require('package')" will try to require package and will fail if package is not available.

• Running node -e "console.log(require.resolve('package'))" (or simply nodepath package) will log the path to the resolved package.

• Adding NODE_DEBUG=module to the above commands will show debug information about module lookup paths. For example NODE_DEBUG=module node -e "require('package')"

Helpful things to know

• Latest versions of node have an "exports" field in package.json. If you're inside the source directory (for example, when running autopkgtest), self-referencing kicks in and exports field is used to resolve package names. Read about self-referencing https://nodejs.org/api/packages.html#packages_self_referencing_a_package_using_its_name

• /var/lib/dpkg/status has the control file information of all installed packages. apt show package can be used to view this information only for one package.

• dpkg -S /path/to/file tells you which package the file at /path/to/file belongs to

• pkg-js-tools is involved a lot. The documentation is at https://salsa.debian.org/js-team/pkg-js-tools/-/tree/master/doc/tools and https://salsa.debian.org/js-team/pkg-js-tools/-/tree/master/doc/autopkgtest

• The "blame" feature on github or git can be used to find which commit introduced a particular line (in package.json, yarn.lock, etc). This might sometimes give clues on why a package suddenly stopped working.

• When a package builds its own typescript definitions (*.d.ts files), dh-nodejs will install them, as long as they are declared in package.json. They won't go to /usr/share/nodejs/@types/thatmodule/, and instead are kept inside /usr/share/nodejs/thatmodule/, and that's perfectly fine.