permanent capture & analysis on linux [not Wireshark-specific]

This is not exactly a Wireshark question, but I wasn't sure where to ask. If you have an idea for a community where to turn to, let me know. In the mean time...

I'm setting up a server/gateway machine (a linux desktop) that is supposed to capture all traffic coming through it via LAN or VPN. And by that I mean write to disk every single bit of network traffic content. The purpose is security and debugging review and analysis.

However, I don't want to just create a gigantic dump file of packets. For it to be useful, it has to:

1) Filter out some white-listed content. So not write to disk stuff like YouTube videos or BitTorrent downloads. (Ergo: white-listing rules by protocol, IPs/domains, header/content keywords, DPI. Also, some rules could alert me in real-time when strange things happen.)

2) Let me analyze that traffic in a useful manner. That is, I'd like to be able to easily query for things like:

• all the files (html, js, png) that my Android Firefox loaded this morning when it crashed; (Assuming I know the time more or less, and the URL I tried to open.)
• all the communications that an Android music app sent home this week, so that I can figure out how it knows my location;
• the update my Ubuntu laptop downloaded yesterday, since from that moment it behaves weirdly.

etc, etc.

How do I go about this? Are there open-source tools that do this?