Project

General

Profile

Bug #17289

FTBFS: git submodules

Added by CyrilBrulebois 4 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Build system
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
feature/15342-cache-wiki
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

Just hit yet another instance of:

+ git clone --shared /amnesia.git/.git/modules/submodules/aufs-standalone
fatal: repository '/amnesia.git/.git/modules/submodules/aufs-standalone' does not exist

(Starting from a vm:destroy'd build environment because of the ENOSPC.)

If that doesn't get fixed for real soon, we should definitely include a hint somewhere about possible workarounds.


Related issues

Related to Tails - Bug #16607: FTBFS with "repository '/amnesia.git/.git/modules/submodules/aufs4-standalone' does not exist" Resolved 03/23/2019

Associated revisions

Revision 32b1ac98 (diff)
Added by intrigeri 3 months ago

Build system: set the permissions that Vagrant needs inside the source tree (refs: #11411, #16607, #17289)

Revision 6684a825 (diff)
Added by intrigeri 3 months ago

Build system: abort early if the permissions on one of the parent directories are too strict (refs: #11411, #16607, #17289)

Revision f5c75d4e
Added by intrigeri 3 months ago

Merge branch 'feature/15342-cache-wiki' into stable (Closes: #15342, #17288, #16607, #17289, #11411)

History

#1 Updated by CyrilBrulebois 4 months ago

And I'm hitting this again, even with everyone under submodules/ with g+w, and after vm:destroy-ing to make sure I was starting from a clean slate.

#2 Updated by CyrilBrulebois 4 months ago

Also checked that chmod -R go+rX .git/modules/submodules wouldn't change anything because those permissions were already there.

Getting rid of the basebox entirely (rake basebox:clean_all) seems to work, but that's a rather big hammer…

#3 Updated by CyrilBrulebois 4 months ago

What's with aufs4-standalone vs. aufs-standalone anyway?

The first build is fine but any subsequent build fails, and I need to start over (going as far as trashing the basebox), which is entirely unreasonable.

kibi@armor:~/work/clients/tails/release/release-checkout$ ls submodules/
aufs-standalone/        chutney/                jenkins-tools/          mirror-pool-dispatcher/ 

kibi@armor:~/work/clients/tails/release/release-checkout$ cat .gitmodules 
[submodule "submodules/jenkins-tools"]
    path = submodules/jenkins-tools
    url = https://git-tails.immerda.ch/jenkins-tools
[submodule "submodules/chutney"]
    path = submodules/chutney
    url = https://git-tails.immerda.ch/chutney
    branch = feature/tails_test_suite
[submodule "submodules/mirror-pool-dispatcher"]
    path = submodules/mirror-pool-dispatcher
    url = https://git-tails.immerda.ch/mirror-pool-dispatcher
[submodule "submodules/aufs-standalone"]
    path = submodules/aufs-standalone
    url = https://github.com/sfjro/aufs5-standalone.git

Meanwhile, on the guest VM:

vagrant@vagrant-buster:/amnesia.git/.git/modules/submodules$ ls -l
total 20
drwxr-xr-x 8 vagrant vagrant 4096 Sep  7 22:23 aufs4-standalone
drwxr-xr-x 8 vagrant vagrant 4096 Dec  2 18:56 chutney
drwxr-xr-x 8 vagrant vagrant 4096 Dec  2 18:56 jenkins-tools
drwxr-xr-x 8 vagrant vagrant 4096 Dec  2 18:56 mirror-pool-dispatcher
drwxr-xr-x 8 vagrant vagrant 4096 Dec  2 09:33 pythonlib

#4 Updated by CyrilBrulebois 4 months ago

It seems getting rid of all the contents of .git/modules, then running:

git submodule deinit -f .
git submodule init
git submodule update

did the trick, and I'm no longer seeing aufs4-standalone, but aufs-standalone instead.

Absolutely no idea why the first build succeeds but not the second/n-th ones… But we might need to have some kind of self-healing process following 7a4dc47c5d2e5aa57e83ddfa40cbf5964cf691cb which renamed aufs4-standalone into aufs-standalone.

Maybe this depends on the git version on the host, which might explain why some developers didn't get this issue (I'm still running stretch, because ENOTIME to upgrade, plus running the test suite without the VM indirection seemed a good reason not to rush anything).

#5 Updated by CyrilBrulebois 4 months ago

In another checkout, this seemed to work:

rm .git/modules/submodules/aufs4-standalone/ -rf
git submodule init
git submodule update

#6 Updated by intrigeri 4 months ago

  • Related to Bug #16607: FTBFS with "repository '/amnesia.git/.git/modules/submodules/aufs4-standalone' does not exist" added

#7 Updated by intrigeri 4 months ago

I'm very sorry the renaming of aufs4-standalone to aufs-standalone causes trouble. Previous aufsN → aufsN+1 renaming caused trouble already, which is why I wanted to do one last renaming and avoid the problem happening again.

I remember this has caused trouble for other people too. I think most developers had to do $something to cope with this renaming since September. I should have provided instructions to do so. By now, I expect all active developers have gone through this transition already and the bulk of the pain is behind us. So it might be too late now to figure out how to document a transition that actually happened in practice?

On the workaround front I can suggest:

  • If you don't do this consistently already: ensure you run git submodule update --init when switching between branches that have different sets of submodules. But I think it is not sufficient to address corner cases such as this one.
  • Sometimes I need to rake vm:ssh and then rm -rf amnesia.git, to workaround weird build failures (the build system does not see the branch I'm trying to build). I've still not taken time to report this, mostly because I am presently unable to describe what are the conditions for this problem to happen. Regardless, I suspect it could have solved the problem you've faced.

#8 Updated by intrigeri 3 months ago

  • Status changed from Confirmed to In Progress
  • Assignee set to intrigeri
  • Feature Branch set to feature/15342-cache-wiki

#9 Updated by intrigeri 3 months ago

  • Status changed from In Progress to Needs Validation
  • Assignee deleted (intrigeri)
  • Target version set to Tails_4.2

#10 Updated by CyrilBrulebois 3 months ago

  • Target version changed from Tails_4.2 to Tails_4.3

#11 Updated by intrigeri 3 months ago

  • Status changed from Needs Validation to Resolved
  • % Done changed from 0 to 100

Also available in: Atom PDF