Project

General

Profile

Bug #16788

Tests fail with "Found something in the pcap file that either is non-IP, or cannot be parsed (RuntimeError)"

Added by intrigeri 6 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Test suite
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
test/16788-fix-pcap-analysis
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

See e.g. https://jenkins.tails.boum.org/view/RM/job/test_Tails_ISO_stable/1829/cucumberTestReport/. I've not looked very closely but the first few test suite jobs I had to look at today exposed this problem.

@anonym, I had never seen it before, could this be related to the recent merge of feature/16148-unfiltered-pcaps?


Related issues

Related to Tails - Bug #16148: ICMPv6 leaks detected by test suite Resolved 11/23/2018
Related to Tails - Bug #16825: "Found something that is not an ethernet packet" intermittent test failure Resolved
Related to Tails - Bug #17102: "Found something that is not an ethernet packet" intermittent test failure Resolved
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

Associated revisions

Revision 86bd6d5d (diff)
Added by anonym 6 months ago

Test suite: don't fail DHCP check for non-IP packets.

Since we started recording the router's traffic in our PCAP
files (refs: #16148) we have started to see this check fail. Indeed,
in some of the captures I've seen the router send an ARP request,
which we don't have a case for. It is sufficient to check only IP
packets, since DHCP only uses IP packets, so we don't have to fail if
we find something else, just ignore them.

Will-fix: #16788

Revision 2df054b8 (diff)
Added by anonym 6 months ago

Test suite: only accept IP/ARP during DHCP check.

While it is true that DHCP only uses IP we might as well be more
thorough. The only non-IP packets that are expected here are ARP
packets, so let's just fail on anything else, like we did prior to
commit 86bd6d5db9284e6204192dae2f7cae1ddaf20df2.

Refs: #16788

Revision 7a4f346a
Added by anonym 6 months ago

Merge remote-tracking branch 'origin/test/16788-fix-pcap-analysis' into stable

Fix-committed: #16788

History

#1 Updated by intrigeri 6 months ago

#2 Updated by intrigeri 6 months ago

  • Related to Bug #16148: ICMPv6 leaks detected by test suite added

#3 Updated by anonym 6 months ago

  • Status changed from Confirmed to In Progress

#4 Updated by anonym 6 months ago

  • % Done changed from 0 to 40
  • Feature Branch set to test/16788-fix-pcap-analysis

intrigeri wrote:

anonym, I had never seen it before, could this be related to the recent merge of feature/16148-unfiltered-pcaps?

Yup. Since we started including all traffic, including the router's, in #16148 I only adapted pcap_connections_helper() but didn't think about this check. It's an easy fix!

I've grepped for pcap and couldn't find any other code that needs adaptation, so we should be good now. Jenkins?

#5 Updated by anonym 6 months ago

  • Status changed from In Progress to Needs Validation
  • Assignee deleted (anonym)
  • % Done changed from 40 to 50

The two DHCP scenarios that are failing very often on all branches are passing again! Let's get this merged!

#6 Updated by intrigeri 6 months ago

  • Assignee set to intrigeri

#7 Updated by intrigeri 6 months ago

  • Status changed from Needs Validation to In Progress
  • Assignee changed from intrigeri to anonym

The reasoning behind this fix implicitly trusts that the system under test behaves as expected ("DHCP only uses IP packets") and then we ignore unexpected behavior. Assuming that a cleaner fix (e.g. ignoring only ARP from the router) is non-trivial, OK, fine, let's do this, but then please explain the reasoning and trade-off in a comment, not just in the commit message: otherwise, next time I'll read this code, I'll be like "uh, seriously?!" :)

#8 Updated by anonym 6 months ago

  • Status changed from In Progress to Needs Validation
  • Assignee changed from anonym to intrigeri
  • % Done changed from 50 to 60

intrigeri wrote:

The reasoning behind this fix implicitly trusts that the system under test behaves as expected ("DHCP only uses IP packets") and then we ignore unexpected behavior.

I see your point. In practice we are not really testing DHCP only, since there' a full system running (hence the unrelated ARP requests), so I think what you propose is likely to cause robustness issues in general, but since this step will only be used in this precise situation (where ARP is the only unrelated thing that can reasonably be expected, AFAICT) I think it is fine, so I pushed something I think you'll like more.

#9 Updated by intrigeri 6 months ago

  • Assignee changed from intrigeri to anonym

I pushed something I think you'll like more.

Indeed, I like it a lot! Feel free to merge yourself once Jenkins is happy :)

#10 Updated by anonym 6 months ago

  • Status changed from Needs Validation to 11
  • Assignee deleted (anonym)
  • % Done changed from 60 to 100

#11 Updated by intrigeri 6 months ago

  • Related to Bug #16825: "Found something that is not an ethernet packet" intermittent test failure added

#12 Updated by intrigeri 5 months ago

  • Status changed from 11 to Resolved
  • Target version changed from Tails_3.15 to Tails_3.14.1

#13 Updated by intrigeri 2 months ago

  • Related to Bug #17102: "Found something that is not an ethernet packet" intermittent test failure added

Also available in: Atom PDF