Formal Quality Assurance of Wireshark

asked 2018-04-16 03:22:23 +0000

Tricon
1 ●1 ●1 ●1

Hello Folks,

I am using Wireshark in the software quality assurance process of a software product under development. One of the standards requirements in the development of the product is to use "certified" tools. Such tools are built following a software quality framework with appropriate evidence.

I am unable to find evidence that Wireshark has a formal QA system. Does it have something?

If it does not, researching in Google Scholar, it appears that evidence of QA within a FOSS environment is difficult to find or a QA process not explicitly followed. So Wireshark is typical of other products within the domain.

When people use Wireshark (or any other FOSS product perhaps) in the QA process of another product under development, how does one provide evidence in the tieback to the tool being developed using appropriate QA process and therefore suitable for use?

Thank you for your time.

edit retag flag offensive close merge delete

Comments

One of the standards requirements in the development of the product is to use "certified" tools. Such tools >are built following a software quality framework with appropriate evidence

What says that any software developed with QA is fit to do a particular job? :-)

If you use Wireshark to verify that your product full fills a protocol standard there is no guarantee that wireshark interprets the protocol "correctly". However we have a number of tests in place to try to assure code quality in the sense of not crashing on bad data various static code analyses run on the code etc. Any code entered has peer review.

Anders ( 2018-04-16 14:17:00 +0000 )edit

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2018-04-20 15:04:26 +0000

Lekensteyn
2346 ●3 ●19 ●29 https://lekensteyn.nl

Wireshark development is largely driven by volunteers, most of them focus only on a narrow area. There are many open issues on the bug tracker (https://bugs.wireshark.org/), but many of them might not be relevant when you use Wireshark to test your product. As others have said, there is code review, automated tests (a small test suite, fuzzing, build tests).

As for how some organizations approach the problem of using a "certified" tool, the Thread Group have developed Wireshark dissectors and use it for in their certification process. For this purpose, they record a specific version of Wireshark that was used for testing to ensure that the results are reproducible. This approach seems reasonable for their needs where they want to ensure that implementations adhere to the standard.

edit flag offensive delete link

Comments

So basically they did QA themselves.

Jaap ( 2018-04-20 16:03:43 +0000 )edit

This is what we did. We took an older GTK-based version with our own dissectors and did our own qualification on that version. Now, when we collect traffic or do a first analysis we will usually use the current version, but when the verification/validation team have a specific test where Wireshark is called for, we will drop to the validated version to provide the documented evidence for their specific test. The results have never conflicted in the versions, but if we use the validated version to provide the results then our process allows for no other documentation is required. I guess if the results would conflict I would have to decide which is correct and deal with the consequences (i.e if validated version is wrong, may need to re-validate, or whatever).

Bob Jones ( 2018-04-20 18:39:16 +0000 )edit

add a comment

answered 2018-04-17 05:26:37 +0000

Jaap

13742 ●710 ●115

I would dare to say that Wireshark has an 'informal QA system'. That means that we combine thorough software engineering practices (code review, automatic builds and tests of the code quality) with an involved community which reflects on the usage aspects of the program, including correct protocol interpretations. This has, over the years, build the confidence in the quality of the program.

edit flag offensive delete link

add a comment