Joy - A Packet For Capturing As Well As Analyzing Network Menstruation Information As Well As Intraflow Data, For Network Research, Forensics, As Well As Safety Monitoring

Joy is a BSD-licensed libpcap-based software bundle for extracting information features from alive network traffic or packet capture (pcap) files, using a flow-oriented model like to that of IPFIX or Netflow, too and then representing these information features inward JSON. It also contains analysis tools that tin travel applied to these information files. Joy tin travel used to explore information at scale, peculiarly safety too threat-relevant data.

JSON is used inward social club to brand the output easily consumable past times information analysis tools. While the JSON output files are somewhat verbose, they are reasonably small, too they response good to compression.

Joy tin travel configured to obtain intraflow data, that is, information too information virtually events that hap inside a network flow, including:

• the sequence of lengths too arrival times of IP packets, upwards to roughly configurable number of packets.
• the empirical probability distribution of the bytes inside the information constituent of a flow, too the entropy derived from that value,
• the sequence of lengths too arrival times of TLS records,
• other non-encrypted TLS data, such every bit the listing of offered ciphersuites, the selected ciphersuite, the length of the clientKeyExchange field, too the server certificate strings,
• DNS names, addresses, too TTLs,
• HTTP header elements too the starting fourth dimension 8 bytes of the HTTP body, and
• the cite of the procedure associated amongst the flow, for flows originate or terminate on the host on which pcap is running.

Joy is intended for usage inward safety research, forensics, too for the monitoring of (small scale) networks to respect vulnerabilities, threats too other unauthorized or unwanted behavior. Researchers, administrators, penetration testers, too safety operations teams tin position this information to expert use, for the protection of the networks beingness monitored, too inward the instance of vulnerabilities, for the practice goodness of the broader community through improved defensive posture. As amongst whatever network monitoring tool, Joy could potentially travel misused; practice non usage it on whatever network of which y'all are non the possessor or the administrator.

Flow, inward positive psychology, is a dry ground inward which a someone performing an action is fully immersed inward a feeling of energized focus, deep involvement, too joy. This minute pregnant inspired the selection of cite for this software package.

Joy is alpha/beta software; nosotros promise that y'all usage it too practice goodness from it, but practice sympathise that it is non suitable for production use.

TLS Fingerprinting

We stimulate got latterly released the largest too most informative opened upwards source TLS fingerprint database. Among other features, our approach builds on previous piece of occupation past times beingness fully automated too annotating TLS fingerprints amongst significantly to a greater extent than information. We stimulate got built a laid of python tools to enable the application of this database, every bit good every bit the generation of novel databases amongst the assist of Joy. For to a greater extent than information, delight run across the TLS fingerprinting documentation.

Relation to Cisco ETA

Joy has helped back upwards the query that paved the means for Cisco’s Encrypted Traffic Analytics (ETA), but it is non straight integrated into whatever of the Cisco products or services that implement ETA. The classifiers inward Joy were trained on a pocket-size dataset several years ago, too practice non stand upwards for the classification methods or performance of ETA. The intent of this characteristic is to permit network researchers to rapidly develop too deploy their ain classifiers on a subset of the information features that Joy produces. For to a greater extent than information on preparation your ain classifier, run across saltUI/README or accomplish out to joy-users@cisco.com.

Credits
This bundle was written past times David McGrew, Blake Anderson, Philip Perricone too Bill Hudson {mcgrew,blaander,phperric,bhudson}@cisco.com of Cisco Systems Advanced Security Research Group (ASRG) too Security too Trust Organization (STO).

Release 4.3.0

• Add IPv6 back upwards to Joy too libjoy
• IPFix collection too export solely back upwards IPv4
• NFv9 solely supports IPv4
• Anonymization solely supports IPv4 addresses
• Subnet labeling solely supports IPv4 addresses

Release 4.2.0

• Re-write joy.c to usage libjoy library
• Updated joy.c to utilize multi-threads for menstruation processing
• Updated unit of measurement tests too python tests to reverberate novel code changes
• Removed guts of the updater procedure to prepare for re-write
• Fixed põrnikas inward processing multiple files on the ascendance line
• Other child põrnikas fixes

Release 4.0.3

• Added back upwards for brand install for Centos

Release 4.0.2

• Add back upwards for fingerprinting

Release 4.0.1
We are pleased to denote the 4.0.1 liberate of the package, which has these features:

• Add additional API's for raise application processing of Flow Records too information features
• Fixed TCP retransmission too out of social club detection
• Better identification of IDP packet
• Fixed roughly retention usage issues
• Fixed child bugs
• Removed dead code

Release 4.0.0
We are pleased to denote the 4.0.0 liberate of the package, which has these features:

• Add back upwards for edifice amongst autotools. ./configure;make clean;make

Release 3.0.0
We are pleased to denote the 3.0.0 liberate of the package, which has these features:

• Modified JOY infrastructure code to travel thread safe.
  • Allowed back upwards multiple piece of occupation threads for packet processing.
  • Each worker thread uses ain output file.
  • Removed global variables for Config.
  • Modified code infrastructure to usage Config Structure.
• Modified the Makefile organization to create the JOY infrastructure every bit a static too shared library.
• Implemented an API for utilizing the JOY Library (joy_api.[hc]).
• Implemented a Vector Packet Processing integration system to utilize VPP native infrastructure when edifice that integration.
• Created 2 API seek out programs, joy_api_test.c too joy_api_test2.c.
• Modified existing seek out programs to link against static JOY library instead of re-compiling the infrastructure code.
• Modified versioning to usage Common Security Module (CSM) conventions.
• Modified build_pkg to pick out bundle version on the ascendance line.
• Cleaned upwards coverity errors too warnings.
• Various põrnikas fixes.

Release 2.0
We are pleased to denote the 2.0 liberate of the package, which has these features:

• The JSON schema has been updated to travel meliorate organized, to a greater extent than readable, too to a greater extent than searchable (by putting searchable keywords every bit the JSON names),
• The novel sleuth tool replaces query/joyq, too brings novel functionality such every bit —fingerprint,
• Much improved documentation, which covers the joy too sleuth tools, examples, too the JSON schema (see using-joy)

Quick Start
Joy has been successfully run too tested on Linux (Debian, Ubuntu, CentOS, too Raspbian), Mac OS X too Windows. The organization has been built amongst gcc too GNU make, but it should piece of occupation amongst other evolution environments every bit well.
Go to the Wiki for a conduct on building: Build Instructions

Download Joy