Project

General

Profile

Bug #12507

Vagrant: too little memory allocated for non-RAM builds

Added by vk over 2 years ago. Updated 10 months ago.

Status:
Rejected
Priority:
Normal
Assignee:
Category:
Build system
Target version:
-
Start date:
05/04/2017
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:
Starter:
Yes
Affected tool:

Description

Hi,

Would it be feasible to have a possibility to customize amount of RAM for build VM?
Right now it's 512M for on-disk builds and 9ish G for RAM builds.
I can submit a patch which will add a possibility to specify this in `TAILS_BUILD_OPTIONS`, similar to `cpus=N`.

Also, I'd like to have ability to point vagrant-libvirt to a custom storage domain. Not everyone is using `default` one.
This can be done either from separate env variable, or via `TAILS_BUILD_OPTIONS`.
Can submit a patch for this as well.

Please let me know.


Related issues

Related to Tails - Feature #12533: Ability to point vagrant-libvirt to a custom storage pool Confirmed 05/10/2017

History

#1 Updated by intrigeri over 2 years ago

  • Assignee set to anonym

anonym, can you please handle the initial triaging of this ticket?

Also, maybe this should be split into 2 different tickets.

#2 Updated by anonym over 2 years ago

  • Assignee changed from anonym to vk
  • QA Check set to Info Needed

vk wrote:

Would it be feasible to have a possibility to customize amount of RAM for build VM?

Sure, but I'm curious why you want this. I mean, currently you achieve the same by bumping the numbers in vagrant/lib/tails_build_settings.rb, and I cannot really come up with a scenario where that is not suitable. If your build fails because there's too little memory around, that number should be bumped permanently in Git, not bumped temporarily. But I might have missed something! :)

I can submit a patch which will add a possibility to specify this in `TAILS_BUILD_OPTIONS`, similar to `cpus=N`.

If you have a legit use case, sure! It might be a good time to rename the ram and noram options to rambuild and diskbuild, so ram=N can be used for this, although this would need some coordination with out sysadmins so they can adjust our infra accordingly.

Also, I'd like to have ability to point vagrant-libvirt to a custom storage domain. Not everyone is using `default` one.
This can be done either from separate env variable, or via `TAILS_BUILD_OPTIONS`.
Can submit a patch for this as well.

This makes a lot of sense, but please file a new ticket and assign it to me. A patch would be nice, but there's some significant rework going on so if you want to do it, please base your patch on the wip/11972-use-vagrant-in-jenkins Git branch. In any case, assign the new ticket to me.

#3 Updated by vk over 2 years ago

anonym wrote:

vk wrote:

Would it be feasible to have a possibility to customize amount of RAM for build VM?

If your build fails because there's too little memory around, that number should be bumped permanently in Git, not bumped temporarily. But I might have missed something! :)

You did not miss anything, this is exactly my use case. With 512MB build was failing, adding 2G swap to the tails-builder VM did fix it; I'm testing with 2048MB / no swap now, will let you know if that's enough.
Another option would be to just have swap partition added to the builder VM, not sure what would be the acceptable solution in your infra context.

If you have a legit use case, sure! It might be a good time to rename the ram and noram options to rambuild and diskbuild, so ram=N can be used for this, although this would need some coordination with out sysadmins so they can adjust our infra accordingly.

If it will be decided to bump the VM_MEMORY_FOR_DISK_BUILDS in vagrant/lib/tails_build_settings.rb - I'll take care about renaming the options too.

This makes a lot of sense, but please file a new ticket and assign it to me. A patch would be nice, but there's some significant rework going on so if you want to do it, please base your patch on the wip/11972-use-vagrant-in-jenkins Git branch. In any case, assign the new ticket to me.

Filed https://labs.riseup.net/code/issues/12533, assigned to you.

#4 Updated by emmapeel over 2 years ago

  • Status changed from New to Confirmed
  • Assignee deleted (vk)
  • QA Check changed from Info Needed to Dev Needed

#5 Updated by emmapeel over 2 years ago

  • Subject changed from Vagrant: ability to adjust RAM, ability to point vagrant-libvirt to custom storage domain to Vagrant: ability to adjust RAM

#6 Updated by intrigeri over 2 years ago

  • Tracker changed from Feature to Bug
  • Subject changed from Vagrant: ability to adjust RAM to Vagrant: too little memory allocated for non-RAM builds
  • Assignee set to vk
  • Type of work changed from Sysadmin to Code

(vk is working on it so they should remain the assignee.)

#7 Updated by anonym over 2 years ago

vk wrote:

anonym wrote:

vk wrote:

Would it be feasible to have a possibility to customize amount of RAM for build VM?

If your build fails because there's too little memory around, that number should be bumped permanently in Git, not bumped temporarily. But I might have missed something! :)

You did not miss anything, this is exactly my use case. With 512MB build was failing, adding 2G swap to the tails-builder VM did fix it; I'm testing with 2048MB / no swap now, will let you know if that's enough.
Another option would be to just have swap partition added to the builder VM, not sure what would be the acceptable solution in your infra context.

Let's bump the RAM instead of adding a swap partition since the RAM requirement still will be pretty low, and swap can be terribly slow if you don't have an SSD.

If you have a legit use case, sure! It might be a good time to rename the ram and noram options to rambuild and diskbuild, so ram=N can be used for this, although this would need some coordination with out sysadmins so they can adjust our infra accordingly.

If it will be decided to bump the VM_MEMORY_FOR_DISK_BUILDS in vagrant/lib/tails_build_settings.rb - I'll take care about renaming the options too.

If disk builds fail due to too little RAM then VM_MEMORY_FOR_DISK_BUILDS definitely should be bumped, so yeah, it's decided! :) So now this ticket is about how much to bump it.

However, note that bumping VM_MEMORY_FOR_DISK_BUILDS will bump VM_MEMORY_FOR_RAM_BUILDS as much, but RAM builds do work so this is undesirable. This means that BUILD_SPACE_REQUIREMENT is too big, and that's very likely; we have been lazy and just bumped BUILD_SPACE_REQUIREMENT for the past years when it probably was the system that needed more RAM for its processes. So, you should decrease BUILD_SPACE_REQUIREMENT with the same amount you bump VM_MEMORY_FOR_DISK_BUILDS.

[Yeah, we never do disk builds, in case you wondered. :)]

Bonus points: it'd be cool if you did a disk build and monitored the disk usage very closesly, e.g. you do a rake vm:ssh and run something like:

while true; do
  df / | tail -n1 | awk '{ print $3 }'
  sleep 0.5
done >> /home/vagrant/disk-usage-monitoring.log

in parallel with the build. Then we can look for the highest value, subtract the first value (i.e. disk usage before the build started) and thus get the peak usage, which then could be used to find a better value for BUILD_SPACE_REQUIREMENT. Of course, we'll want a higher value, to leave some marigin.

#8 Updated by u about 2 years ago

Ping?

#9 Updated by u over 1 year ago

  • Related to Feature #12533: Ability to point vagrant-libvirt to a custom storage pool added

#10 Updated by u over 1 year ago

Are you still interested in working on this? If not, please deassign yourself from this ticket. It will make ticket tracking/triaging easier for us. Thanks.

#11 Updated by intrigeri 10 months ago

  • Status changed from Confirmed to Rejected

@vk, please let us know if you're still interested and I'll reopen this ticket :)

Also available in: Atom PDF