Project

General

Profile

Feature #7427

Feature #5926: Freezable APT repository

Feature #9487: Research what solution to use for the freezable APT repository

Evaluate using aptly

Added by intrigeri over 5 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Infrastructure
Target version:
Start date:
06/21/2014
Due date:
% Done:

100%

Feature Branch:
Type of work:
Test
Blueprint:
Starter:
No
Affected tool:

Description

Using aptly for managing the packages imported from foreign APT repository may be a good alternative to reprepro, if #6295 doesn't give good results, and if the grml's solution is not suitable.

History

#1 Updated by intrigeri over 5 years ago

  • Tracker changed from Bug to Feature

#2 Updated by intrigeri over 5 years ago

  • Blocked by Feature #6906: Ask the Grml people how they handle the "keep lots of source packages around" problem added

#3 Updated by intrigeri over 5 years ago

  • Blocked by Feature #6295: Evaluate consequences of importing large amounts of packages into reprepro added

#4 Updated by intrigeri over 5 years ago

aptly is now in Debian sid.

#5 Updated by intrigeri over 4 years ago

  • Blocked by deleted (Feature #6295: Evaluate consequences of importing large amounts of packages into reprepro)

#6 Updated by intrigeri over 4 years ago

  • Blocked by deleted (Feature #6906: Ask the Grml people how they handle the "keep lots of source packages around" problem)

#7 Updated by intrigeri over 4 years ago

  • Assignee set to intrigeri
  • Target version changed from Sustainability_M1 to Tails_2.3
  • Parent task set to #9487

aptly 0.8 is in Jessie.

#9 Updated by intrigeri over 4 years ago

  • Blocked by Feature #6295: Evaluate consequences of importing large amounts of packages into reprepro added

#10 Updated by intrigeri over 4 years ago

  • Blocked by Feature #6906: Ask the Grml people how they handle the "keep lots of source packages around" problem added

#11 Updated by intrigeri over 4 years ago

  • Target version changed from Tails_2.3 to 246

#12 Updated by intrigeri over 4 years ago

The aptly feature set, semantics and user interface looks very close to what (I believe) we need:

  • aptly mirror create mirrors a remote repo; one can select distributions, components, and architectures; and then aptly mirror update actually downloads stuff. Note that:
    • Apparently the filter feature doesn't support reading a file for input, so it's not suitable for listing lots of packages => need to mirror entire distributions (which should be simpler anyway, by avoiding having to deal e.g. with topic branches pulling in more packages).
    • By default, aptly merges components into a single one (do we want to do that?), but it can also do multi-component publishing
    • So far, aptly always re-signs mirrored indices. Re-publishing them as-is, with the original signature, is part of upstream's plans. Now, of course this won't be compatible with apt snapshot pull or any other operation that modifies a snapshot.
  • aptly snapshot create creates a named snapshot from the current state of a mirror or local repo; the snapshot can then be published with aptly publish snapshot
  • aptly snapshot pull allows us to pull selected packages (optionally, with their dependencies) from a mirror into an existing (immutable) snapshot, creating a new snapshot as a result; and then aptly publish switch updates a published repo to a new snapshot. The combination of those is probably what we need in order to pull selected package upgrades from Debian during a freeze.

=> even if reprepro would work as well, I must say that aptly is very tempting. In particular, separating our own APT repo from mirrors/snapshots of remote repos is appealing: it decreases chances that the one affects the other (e.g. automatic cleanup operations).

Also:

  • aptly generates a directory structure that can be served as-is with any web server.
  • aptly task can chain commands into a single one, which can be useful when writing wrappers. It's said to be experimental, though.
  • There's an API and a CLI tool to interact with it
  • There's at least one Puppet module for the initial setup and very basic functionality (mirror and local repo creation). It can also manage the aptly API's web service, but currently only supports upstream.

#13 Updated by intrigeri over 4 years ago

  • Blocked by deleted (Feature #6295: Evaluate consequences of importing large amounts of packages into reprepro)

#14 Updated by intrigeri over 4 years ago

  • Blocked by deleted (Feature #6906: Ask the Grml people how they handle the "keep lots of source packages around" problem)

#15 Updated by intrigeri over 4 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10
  • Type of work changed from Sysadmin to Test

Next step is to actually test the typical workflow we would use.

#17 Updated by intrigeri over 4 years ago

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • Target version changed from 246 to Tails_1.7
  • % Done changed from 10 to 100

We've quickly hit one problem that reflects what I see as a blocker (lack of maturity): the generated Release files have no Valid-Until field. The list of bugs, including ones that cause data loss, we've seen on GitHub, is a bit scary. We've decided to stick to reprepro for now. Too bad, the interface looked very nice.

Also available in: Atom PDF