No description
Find a file
Reepca Russelstein 78da695178
daemon: Explicitly unlock output path in the has-become-valid case.
Fixes <https://issues.guix.gnu.org/31785>.

Similar to <https://github.com/NixOS/nix/issues/178>, fixed in
<29cde917fe>.

We can't rely on Goal deletion to release our locks in a timely manner.  In
the case in which multiple guix-daemon processes simultaneously try producing
an output path path1, the one that gets there first (P1) will get the lock,
and the second one (P2) will continue trying to acquire the lock until it is
released.  Once it has acquired the lock, it checks to see whether the path
has already become valid in the meantime, and if so it reports success to
those Goals waiting on its completion and finishes.  Unfortunately, it fails
to release the locks it holds first, so those stay held until that Goal gets
deleted.

Suppose we have the following store path dependency graph:

          path4
      /     |     \
   path1   path2   path3

P2 is now sitting on path1's lock, and will continue to do so until path4 is
completed.  Suppose there is also a P3, and it has been blocked while P1
builds path2.  Now P3 is sitting on path2's lock, and can't acquire path1's
lock to determine that it has been completed.  Likewise P2 is sitting on
path1's lock, and now can't acquire path2's lock to determine that it has been
completed.  Finally, P3 completes path3 while P2 is blocked.

Now:

- P1 knows that path1 and path2 are complete, and holds no locks, but can't
  determine that path3 is complete
- P2 knows that path1 and path3 are complete, and holds locks on path1 and
  path3, but can't determine that path2 is complete
- P3 knows that path2 and path3 are complete, and holds a lock on path2, but
  can't determine that path1 is complete

And none of these locks will be released until path4 is complete.  Thus, we
have a deadlock.

To resolve this, we should explicitly release these locks as soon as they
should be released.

* nix/libstore/build.cc
  (DerivationGoal::tryToBuild, SubstitutionGoal::tryToRun):
  Explicitly release locks in the has-become-valid case.
* tests/store-deadlock.scm: New file.
* Makefile.am (SCM_TESTS): Add it.

Change-Id: Ie510f84828892315fe6776c830db33d0f70bcef8
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2024-12-30 00:51:57 +01:00
.mumi
build-aux build: Restrict access to daemon socket in tests. 2024-10-24 14:50:09 +02:00
doc reconfigure: Support loading the system for kexec reboot. 2024-12-28 22:47:34 +01:00
etc news: Add 'de' translation. 2024-12-29 16:52:25 +01:00
gnu gnu: ecl-command-line-args: Disable tests. 2024-12-29 16:05:28 -06:00
guix reconfigure: Support loading the system for kexec reboot. 2024-12-28 22:47:34 +01:00
m4 build: Fix detection of guile-ssh. 2024-12-12 23:23:33 +01:00
nix daemon: Explicitly unlock output path in the has-become-valid case. 2024-12-30 00:51:57 +01:00
po nls: Update translations. 2024-11-30 10:04:16 +01:00
scripts
tests daemon: Explicitly unlock output path in the has-become-valid case. 2024-12-30 00:51:57 +01:00
.dir-locals.el .dir-locals.el: Configure indentation for home-environment. 2024-11-29 15:45:18 +01:00
.editorconfig
.gitattributes
.gitignore
.guix-authorizations .guix-authorizations: Authorise Greg Hogan's key. 2024-12-22 01:00:00 +01:00
.guix-channel
.mailmap Change email of Evgeny Pisemsky. 2024-08-05 00:44:43 +03:00
.patman
AUTHORS
bootstrap maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'. 2024-04-19 16:45:41 +02:00
ChangeLog
CODE-OF-CONDUCT
config-daemon.ac daemon: Fix linking gcrypt when --as-needed linker arg is used 2024-12-09 23:15:45 +01:00
configure.ac build: Use the po4a command for the translation generation. 2024-06-03 00:27:11 +02:00
COPYING
gnu.scm
guix.scm
HACKING
Makefile.am daemon: Explicitly unlock output path in the has-become-valid case. 2024-12-30 00:51:57 +01:00
manifest.scm maint: Add installer dependencies to the manifest. 2024-11-11 07:28:34 +01:00
NEWS
README
ROADMAP
THANKS
TODO

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

-*- mode: org -*-

[[https://www.gnu.org/software/guix/][GNU Guix]] (IPA: /ɡiːks/) is a purely functional package manager, and
associated free software distribution, for the [[https://www.gnu.org/gnu/gnu.html][GNU system]].  In addition
to standard package management features, Guix supports transactional
upgrades and roll-backs, unprivileged package management, per-user
profiles, and garbage collection.

It provides [[https://www.gnu.org/software/guile/][Guile]] Scheme APIs, including a high-level embedded
domain-specific languages (EDSLs) to describe how packages are to be
built and composed.

GNU Guix can be used on top of an already-installed GNU/Linux distribution, or
it can be used standalone (we call that “Guix System”).

Guix is based on the [[https://nixos.org/nix/][Nix]] package manager.


* Requirements

If you are building Guix from source, please see the manual for build
instructions and requirements, either by running:

  info -f doc/guix.info "Requirements"

or by checking the [[https://guix.gnu.org/manual/en/html_node/Requirements.html][web copy of the manual]].

* Installation

See the manual for the installation instructions, either by running

  info -f doc/guix.info "Installation"

or by checking the [[https://guix.gnu.org/manual/en/html_node/Installation.html][web copy of the manual]].

* Building from Git

For information on building Guix from a Git checkout, please see the relevant
section in the manual, either by running

  info -f doc/guix.info "Building from Git"

or by checking the [[https://guix.gnu.org/manual/en/html_node/Building-from-Git.html][web_copy of the manual]].

* How It Works

Guix does the high-level preparation of a /derivation/.  A derivation is
the promise of a build; it is stored as a text file under
=/gnu/store/xxx.drv=.  The (guix derivations) module provides the
`derivation' primitive, as well as higher-level wrappers such as
`build-expression->derivation'.

Guix does remote procedure calls (RPCs) to the build daemon (the =guix-daemon=
command), which in turn performs builds and accesses to the store on its
behalf.  The RPCs are implemented in the (guix store) module.

* Contact

GNU Guix is hosted at https://savannah.gnu.org/projects/guix/.

Please email <help-guix@gnu.org> for questions and <bug-guix@gnu.org> for bug
reports; email <gnu-system-discuss@gnu.org> for general issues regarding the
GNU system.

Join #guix on irc.libera.chat.

* Guix & Nix

GNU Guix is based on [[https://nixos.org/nix/][the Nix package manager]].  It implements the same
package deployment paradigm, and in fact it reuses some of its code.
Yet, different engineering decisions were made for Guix, as described
below.

Nix is really two things: a package build tool, implemented by a library
and daemon, and a special-purpose programming language.  GNU Guix relies
on the former, but uses Scheme as a replacement for the latter.

Using Scheme instead of a specific language allows us to get all the
features and tooling that come with Guile (compiler, debugger, REPL,
Unicode, libraries, etc.)  And it means that we have a general-purpose
language, on top of which we can have embedded domain-specific languages
(EDSLs), such as the one used to define packages.  This broadens what
can be done in package recipes themselves, and what can be done around them.

Technically, Guix makes remote procedure calls to the nix-worker
daemon to perform operations on the store.  At the lowest level, Nix
“derivations” represent promises of a build, stored in .drv files in
the store.  Guix produces such derivations, which are then interpreted
by the daemon to perform the build.  Thus, Guix derivations can use
derivations produced by Nix (and vice versa).

With Nix and the [[https://nixos.org/nixpkgs][Nixpkgs]] distribution, package composition happens at
the Nix language level, but builders are usually written in Bash.
Conversely, Guix encourages the use of Scheme for both package
composition and builders.  Likewise, the core functionality of Nix is
written in C++ and Perl; Guix relies on some of the original C++ code,
but exposes all the API as Scheme.

* Related software

  - [[https://nixos.org][Nix, Nixpkgs, and NixOS]], functional package manager and associated
    software distribution, are the inspiration of Guix
  - [[https://www.gnu.org/software/stow/][GNU Stow]] builds around the idea of one directory per prefix, and a
    symlink tree to create user environments
  - [[https://www.pvv.ntnu.no/~arnej/store/storedoc_6.html][STORE]] shares the same idea
  - [[https://live.gnome.org/OSTree/][GNOME's OSTree]] allows bootable system images to be built from a
    specified set of packages
  - The [[https://www.gnu.org/s/gsrc/][GNU Source Release Collection]] (GSRC) is a user-land software
    distribution; unlike Guix, it relies on core tools available on the
    host system