opam on Github

opam 2.6.0 release
On , by Raja Boujbel - OCamlPro, Kate Deplaix - Ahrefs, Nathan Rebours - OCamlPro and David Allsopp - Jane Street

Feedback on this post is welcome on Discuss!

We're happy to announce the release of opam 2.6.0 and encourage all users to upgrade. Please read on for installation and upgrade instructions.

Try it!

The upgrade instructions are unchanged:

  1. Either from binaries: run

For Unix systems

bash -c "sh <(curl -fsSL https://opam.ocaml.org/install.sh) --version 2.6.0"

or from PowerShell for Windows systems

Invoke-Expression "& { $(Invoke-RestMethod https://opam.ocaml.org/install.ps1) } -Version 2.6.0"

or download manually from the Github "Releases" page to your PATH.

  1. Or from source, manually: see the instructions in the README.

You should then run:

opam init --reinit -ni

Note: the following section will recap the various major changes in opam 2.6.0 for anyone who haven't already read the previous pre-release announcements. For those who did, note that nothing changed between 2.6.0~rc1 and the final 2.6.0.

Major change: in-place env hook

For people using the shell hooks, this release changed the way PATH is kept up-to-date. Previously, opam took priority over any other elements of PATH by making sure to always be in front. However this causes problem for users expecting manual export PATH=some-custom-bindir:$PATH to prioritise their directory. To resolve this problem, the shell hook now instead replaces the directory managed by opam in-place, keeping the order asked by the user.

To benefit from this, make sure opam init --reinit -ni was ran once after upgrading to this version (automatically done by our install script if it detects an existing opam installation).

Thanks to @gridbugs for this contribution.

Major change: reduce the disk space usage of opam

When installing a package, opam doesn't exactly go easy on disk usage. For people with limited disk space it is a problem which can result in a "no space left on device" type error. While no-one really can get rid of this type of error completely, this release comes with some quite substential improvements to the disk space used during installs.

In particular the build directory is now deleted as soon as possible during a build instead of waiting until the end. We also used to cache both the extracted sources and the original archive of packages. However this is redundant and inefficient on some file-systems, thus opam no longer keep the extracted sources when installing the package.

While the disk usage used by opam can be reduced over time while simply reinstalling packages, you can liberate some free GB in one go using opam clean --all-switches.

Major change: performance improvements on certain file-systems (e.g. NTFS on Windows or IO constrained machines)

Some file-systems, such as NTFS on Windows famously take forever for commands such as opam update, opam init or any command upon upgrading opam. This is due to how opam stored its repositories: i.e. opam-repository is just a large number of directories and files that opam has to scan through to get the informations it needs to run. Once a repository is read once it's usually not a problem anymore as opam caches it internally, however everytime opam update is called the new repository has to be re-read, which on platforms such as Windows or a busy VPS causes long delays because the files and directories being read are not in the OS's internal cache anymore.

To remedy this while keeping the same repository format, we now leverage the existing index.tar.gz file expected to be served by HTTP opam repositories and simply don't extract it. Instead we now use the ocaml-tar library to read the file in-memory, thus only needing one read syscall (against tens of thousands previously, counting on OS-level caches to be fast).

While this only helps HTTP repositories (e.g. the default opam-repository), other types of repositories are usually either smaller (local repositories) or less impacted (VCS repositories) and overall less used than the default HTTP repository so this is less of an issue. However we will still look into it in the future. You can enable this mechanism for all non-VCS repositories by setting the environment variable OPAMREPOSITORYTARRING=1, however this is not as efficient as for HTTP, so it may or may not be worth doing depending on your file-system.

Other noteworthy changes

  • Add root and rootexec sections to .install files to install files from the root prefix. Thanks to @WardBrian for this contribution.

  • Reorder the list of actions by increased priority

  • Improved depexts handling by caching system package availability during opam update, avoiding redundant system checks at install time

  • Allow detection of installed system packages through their virtual names on ALT Linux, RHEL-based and SUSE-based distributions

  • Added --ignore-available-on option to allow ignoring the available: field of certain packages. Thanks once-again to @WardBrian for this contribution.

  • Fix an opam 2.5 regression where opam pin list failed abruptly when the source of the pinned package doesn't exist

  • opam update now supports updating a repository that changed a file to a directory of the same name and vice versa

  • Do not fail on directories named opam when scanning the packages directory of a repository during opam repo add or opam init (worked on subsequent opam update)

  • Fix "undefined variable" error when a lock file filter contains an undefined variables: fail gracefully with strict mode, continue and default the variable to false otherwise

  • Fix opam lock support of dependency formula that include disjunctions

  • Fix package installation during opam pin add <url to archive>. Thanks to @zoggy for this contribution.

  • Fix the filename check used when parsing arguments to allow / to be recognised as a directory separator on Windows

  • Make git calls more deterministic regardless of the global or system config

  • Read full lines when asking for user input when TERM=dumb (e.g. emacs' M-x shell). Thanks to @arvidj for this contribution.

  • opam init --reinit now regenerate the list of valid switches, fix switch internal data (cache, config, packages)

  • Safe mode doesn't reset debuglevel to 0 anymore. Consider updating your scripts to discard stderr or add --debug-level=0 if your script isn't resistant to output on stderr

  • To avoid git underlying maintenance operation from interfering with opam (possible race condition), opam now disable git gc/maintenance on repositories it maintains

Various performance and other improvements were made and bugs were fixed. This release also includes a handful of improvement the documentation and more than two dozen improvement and extensions to our testsuite. API changes are also denoted in the release notes:

Please report any issues to the bug-tracker.

Happy hacking!

Special thanks to the Haematology department and Bone Marrow Transplant Unit of the NHS Greater Glasgow for making this release possible <3

Generated from opam-2-6-0.mdusing opam2web, courtesy of OCamlPro. Commercial support.