Open Vulnerability Assessment Language: An Overview | Lucideus Research
What is OVAL?
OVAL (Open Vulnerability Assessment Language) is an international information security community standard to assess and report upon the machine state of computer systems, promote open and publicly available security content and to also transfer this information across the entire spectrum of security tools and services.
It is an open community effort which aims to standardize the process by which security tool creators, operating system vendors and security professionals test systems for vulnerabilities. It is sponsored by the office of Cybersecurity and Communications at the U.S. Department of Homeland Security. Center for Internet Security manages the OVAL website, community engagement, and discussion lists to enable open and public collaboration within the entire information security community. This was earlier done by MITRE (till July 31, 2015)
Why OVAL?
Until OVAL there was no common or structured means for system administrators and other end users to determine the existence of software vulnerabilities, configuration issues, programs, and/or patches in local systems. Much of the information was available as text-based descriptions from vulnerability and other knowledge sources such as software vendors, government agencies, tool vendors, and security consulting firms, however, it remained a labor-intensive and error-prone process for system administrators to read and interpret this unstructured information and make a determination of whether a particular vulnerability or configuration issue existed on a local system.
OVAL solves these problems by providing standard definitions to determine which software vulnerabilities, configuration issues, programs, and patches exist on a system. Users can then obtain appropriate fix information and software patches from vendors and make the repairs.
How OVAL works?
OVAL consists of three main components. They are:
OVAL Language
The OVAL Language is the most important component of OVAL. It standardizes the three main steps of the assessment process: representing configuration information of systems for testing; analyzing the system for the presence of the specified machine state and reporting the results of this assessment.
How it does this is by having three standardized schemas using which three corresponding XML files are generated. These XML files along with their schemas are as follows:
It is an open community effort which aims to standardize the process by which security tool creators, operating system vendors and security professionals test systems for vulnerabilities. It is sponsored by the office of Cybersecurity and Communications at the U.S. Department of Homeland Security. Center for Internet Security manages the OVAL website, community engagement, and discussion lists to enable open and public collaboration within the entire information security community. This was earlier done by MITRE (till July 31, 2015)
Why OVAL?
Until OVAL there was no common or structured means for system administrators and other end users to determine the existence of software vulnerabilities, configuration issues, programs, and/or patches in local systems. Much of the information was available as text-based descriptions from vulnerability and other knowledge sources such as software vendors, government agencies, tool vendors, and security consulting firms, however, it remained a labor-intensive and error-prone process for system administrators to read and interpret this unstructured information and make a determination of whether a particular vulnerability or configuration issue existed on a local system.
OVAL solves these problems by providing standard definitions to determine which software vulnerabilities, configuration issues, programs, and patches exist on a system. Users can then obtain appropriate fix information and software patches from vendors and make the repairs.
How OVAL works?
OVAL consists of three main components. They are:
- OVAL Language
- OVAL Interpreter
- OVAL Repository
OVAL Language
The OVAL Language is the most important component of OVAL. It standardizes the three main steps of the assessment process: representing configuration information of systems for testing; analyzing the system for the presence of the specified machine state and reporting the results of this assessment.
How it does this is by having three standardized schemas using which three corresponding XML files are generated. These XML files along with their schemas are as follows:
- Definitions file based on the OVAL Definition Schema
- System Characteristics file based on the OVAL System Characteristics schema
- Results file based on the OVAL Results schema
OVAL Definition Schema:
The OVAL Definition schema is used to define the XML framework for writing:
- OVAL Vulnerability Definitions by defining the conditions that must exist on a computer for a specific vulnerability to be present.
- OVAL Patch Definitions by defining the conditions on a computer that determine whether a particular patch is appropriate for a system.
- OVAL Inventory Definitions by defining the conditions on a computer that determine whether a specific piece of software is installed on the system.
- OVAL Compliance Definitions by defining the conditions on a computer that determine compliance with a specific policy or configuration statement.
Let’s take a look at a single definition below. It is extracted from the definitions file for the Windows 7 operating system.
Each definition is distinguished by a unique OVAL Identifier. OVAL-IDs use the format "oval:Organization_DNS_Name:ID_Type:ID_Value" where organization DNS Name is of the form ‘org.mitre.oval’, ID Type denotes the entity to which the ID is being applied (and can be one of the following values: def - Definition, obj - Object, ste - State, tst - Test, or var - Variable), and ID Value is an integer that is unique to the DNS name and ID Type pair that precedes it. For Example: oval:org.mitre.oval:def:8617.
So this is an OVAL Vulnerability Definition as evident by the class field. This definition checks whether Microsoft Office Excel Record Memory Corruption Vulnerability is present or not. All the affected platforms are mentioned along with the affecting product (Microsoft Excel 2002).
The reference field tells us what the source/basis of the definition is. The definition also provides a timestamp of when the definition was submitted, by whom and the various dates in its acceptance. It also shows the timestamp for any modifications that occurred.
Lastly, we have the criteria. This criteria decides if the specified vulnerability is present.
Here, the criteria consists of two criterion which are joined by the AND operator. What this means is that for the vulnerability to be present, both the given criterion need to be true. The first criterion is an extension of another definition and the second one checks if
the excel version is less than 10.0.6860.0. It uses a test to do so. Going to the test we get the following test definition.
Firstly, observe the test id is the same. The comment mentions what the test does. The check_existence attribute specifies how many items in the set defined by the OVAL Object must exist for the test to evaluate to true. The value for this attribute here is 'at_least_one_exists' indicating that the test may evaluate to true if at least one item defined by the OVAL Object exists on the system.
Now looking at the object and state elements we see that the object stores the version of EXCEL.EXE through another variable.
The state on the other hand stores values of all versions below 10.0.6860.0.
Hence, if the value of the object element matches any values in the state versions, then the test returns true. Note: xmlns is the namespace used to identify schema file to validate content.
OVAL System Characteristic Schema:
The OVAL System Characteristics schema defines a standard XML format for representing system configuration information. This configuration information includes OS parameters, installed software application settings, and other security relevant configuration values.
Using this schema, a System Characteristic XML file is generated which is a "database" of system characteristics against which the OVAL Definitions can be compared in order to analyze a system for vulnerabilities, configuration issues, and patches. This XML file is generated when the OVAL interpreter runs on a machine and captures all the required machine state.
In essence, the schema defines a standard system characteristics exchange format which can facilitate very easy exchange of data within the community and between security administrators and vendors.
The XML file generated looks something like this:
Above is a snapshot a System Characteristic XML file, generated when OVAL interpreter ran on an XP machine.
OVAL Results Schema:
The OVAL Results schema defines a standard XML format for storing the results of an evaluation of a system. Using this schema, a Results XML file is generated containing current state of a system’s configuration as compared against a set of OVAL Definitions. This file is generated when OVAL interpreter runs on a machine.
This data then allows users to take the necessary actions to mitigate the vulnerabilities and configuration issues (for example, install patches, alter system configuration settings, and/or take external precautions to limit access to the affected systems).
Below is a snapshot of the Results XML file in .html format:
All the schemas: OVAL Definition schema, OVAL System Characteristics schema and OVAL Results schema, are not individual schemas unto themselves but are each composed of a "core" schema and a collection of "component" schemas.
OVAL Interpreter
The OVAL Interpreter is a freely available reference implementation that demonstrates the evaluation of OVAL Definitions. Based on a set of XML Definitions the Interpreter collects system information like: version of installed softwares, permissions granted to different softwares, access type etc. and stores it in the System Characteristic XML file. It then evaluates it, comparing it against the definitions and thus generates the detailed OVAL Results file. In the example we discussed above, the Interpreter collected the object’s value – the Excel version and stored it in the System Characteristic file. Then it ran the test, checking if the object’s value matches any value in the state.
Developed to demonstrate the usability of OVAL Definitions and to ensure correct syntax and adherence to the OVAL Schemas by definition writers, it is not a fully functional scanning tool and has a simplistic user interface, but running the Interpreter will provide a list of OVAL Definition IDs and their results on the system.
Below is a snapshot of the OVAL Interpreter output file when we ran it on an XP machine:
OVAL Repository
The OVAL Repository is the central meeting place for the OVAL Community to discuss, analyze, store, and disseminate OVAL definitions. OVAL definitions are available here for most major platforms. You can search definitions through various filters like ID, Title, Description, Platforms etc. You can also download the entire repository for a platform.
Members of the community contribute definitions by posting them to the OVAL Repository Forum email discussion list, where the OVAL Team and other members of the community review and discuss them. Any member of the OVAL Community which includes the OVAL Board, organizations with OVAL-Compatible information security products and services, and individual members of the OVAL Repository Forum email list can submit OVAL Definitions.
The Repository Forum is a lightly moderated public discussion list for those interested in writing, submitting, and discussing new and previously posted definitions, as well as the vulnerabilities and configuration issues themselves that affect definition writing.
Below is a picture summarizing the entire process:
Applications
1) Security Advisory Distribution:
Security advisories are published by vendors and security researchers as vulnerabilities are discovered. It contains the information needed to detect the presence of the vulnerability. If the vulnerability information is published in a standard, machine-readable format, it will provide scanning products with immediate access to actionable content that can be used to assess the security posture of a system. OVAL can come quite handy in this.
Suppose in a scenario, a software vendor receives a report of an undisclosed vulnerability and confirms that there is a vulnerability in their software. The vendor further investigates the vulnerability to determine what versions of the software are affected and on what platforms.
The vendor reserves a CVE Identifier for the vulnerability and creates a standardized check for the vulnerability in the form of an OVAL Definition. The software vendor then adds tests to check for the vulnerable software on the relevant platforms. Organizations can download security advisor’s OVAL Definition and use it in their vulnerability scanning tool of choice to determine whether or not their systems are vulnerable.
2) Vulnerability Assessment:
Vulnerability management is the process of identifying the vulnerabilities in a system and prioritizing them according to their severity. Currently, organizations that develop vulnerability management products employ a team of content developers who create customized checks for each vulnerability to indicate whether it is present or not.
OVAL can be of tremendous use to vulnerability management product vendors as they can share vulnerability checks with each other and collaborate on developing the best possible check for a given issue. For example: take a case where a new critical vulnerability is disclosed by an application vendor. A vulnerability management product vendor quickly develops and distributes an OVAL Definition with a check for the presence of the vulnerability on the platforms it supports.
The vendor also shares this new check with a forum of other vulnerability management vendors and industry experts in the form of an OVAL Definition. The OVAL Definition is checked for false positives by security experts and extended by other vendors to include detailed checking information for additional platforms in order to make the vulnerability check complete for all known vulnerable platforms. Thus forum members have collaborated in developing a thorough, accurate, standardized check for the vulnerability and leveraged the resulting OVAL Definition in their products and services.
3) Patch Management:
Patch management is the process of identifying the security issues and software updates that affect a system, applying the patches that resolve these issues, and verifying that the patches were successfully installed.
The patch management vendor community can greatly benefit from having patch checking information structured in a standard format as it will allow them to quickly consume data from multiple sources like different vendors.
If operating system vendors release a new set of patches for their platform and include standardized patch checks as OVAL Definitions then it becomes quite easy to install patches. System administrator just have to run the patch management tool which retrieves the OVAL Definitions, verifies its signature, collects the attributes, expresses the findings in the OVAL Results. The patch management tool then examines the OVAL Results and determines that a patch should be installed. The patch is installed and the system is no longer vulnerable.
4) Configuration Management:
The process of configuration management involves examining a machine’s configuration state, comparing it against a known good or mandated configuration state, and reporting the results. There are a number of publicly available best practice configuration guides but many of these exist in paper form only, and it is up to the IT staff to translate the document into something that can be applied and enforced on a consistent basis. So if vendors release configuration policies in a standardized format with a collection of OVAL Definitions, it becomes extremely easy for customers to use them and customize them further to meet their needs.
Organization deciding to develop a secure configuration policy for its desktop systems can leverage the secure configuration guidance in OVAL definitions recommended by the desktop operating system vendor rather than creating a new policy from scratch. The organization downloads the policy and tailors it to their environment. Suppose, the organization has a very strict password policy and needs to require a minimum password length of 14 characters on all desktop systems. Given that the operating system vendor recommended a minimum password length of 8 characters, there is already an OVAL Definition in the published secure configuration for checking minimum password length. The organization is able to simply tailor the minimum password length value setting to 14 and reuse the rest of the checking logic in the OVAL Definition. The organization can apply several other customizations to the policy by editing the published OVAL Definitions.
5) Auditing and Centralized Audit Validation:
Audit validation provides reports about machine states at any given time in the past. For this, capturing machine configuration information at a high level of granularity is very important. Also this data needs to be stored in a standardized, data-centric format, ensuring that it is not bound to a specific product. An example of how OVAL is useful here is given below.
Organizations have a centralized audit validation system. When a new system joins the network, it is immediately scanned and the OVAL Results, associated with the scans, serve as the system’s initial state in the centralized audit validation system. From then onwards, the system is scanned once a week to determine if anything has changed. If changes are discovered the validation system is updated with the latest OVAL Results.
If at any point in time between the scheduled scans, the system falls into or out of compliance or a new patch has been installed, the centralized audit validation system is immediately updated with the new OVAL Results. Thus the centralized audit validation system now contains multiple sets of OVAL Results documenting the various state changes to the system over time.
6) Security Information Management Systems (SIMS):
SIMS integrate the output of a variety of security, auditing, and configuration products to build a comprehensive view of the security posture of an organization’s network. The fewer data formats the SIM needs to understand the more flexible and powerful the product can be. Hence, standardizing the data exchange formats with OVAL can greatly simplify the interoperability requirements.
For example: a security information management system vendor can utilize the OVAL Results generated by vulnerability management tools, patch management tools, configuration management tools, and any other tool that produces OVAL Results as a primary format for data coming into their system. By doing so, the system can consume data from an entire range of tools in a straightforward manner without the need to translate different formats, of like data, into a single format before it can be analyzed.
7) System Inventory:
System inventory is the process of gathering a detailed listing of the applications installed on a given system. Large enterprises often have many versions of many applications running on wide variety of operating systems. This raises a considerable challenge when tracking softwares for licensing, vulnerability management, compliance, and other purposes.
Having a standardized way to check for the presence of an application and recording its versions is a very important and acknowledged need. System administrators can use OVAL definitions to run against their system inventory tools. This will record all the required system attributes and generate a system configuration file which is a snapshot for the observed state of the system.
8) Malware and Threat Indicator Sharing:
Incident coordination centers, organizations, and other members of the security community actively discuss malware and share low-level system details that can be used to detect potentially compromised systems. These details are commonly shared as prose documents that require translation into actionable content prior to being used for system assessment.
Thus using a standard OVAL format to encode malware and threat indicators into OVAL definitions is very useful. It can help in sharing threat indicators and detecting compromised systems.
Industry Products:
Many companies and organizations have adopted OVAL under the OVAL adoption program. The OVAL Adoption Program is based on five different capabilities, each targeting a different usage of the OVAL Language.
Companies develop products using OVAL and then categorize these products under one or more of the five categories depending on the capabilities of the product. Given below is a list of the categories:
1) Authoring Tool:
Products registered as Authoring Tools should aid in the process of creating new OVAL files (including products that consolidate existing OVAL Definitions into a single file). Vendors utilize an Authoring Tool to assist in vulnerability scanning and the development of patch applicability content to provide to their customers.
2) System Characteristics Producer:
Products registered as a system characteristic producer generate a valid OVAL System Characteristics file based on the details of a system. This can further be used by vendors and organizations alike to generate system images.
3) OVAL Definition Evaluator:
These products use OVAL definitions and system characteristics to guide evaluation and produce an OVAL Results document as output. This can be used to implement Definition Evaluator capability on security advisories and report any vulnerabilities found
4) OVAL Results Consumer:
This product that accepts an OVAL Results document as input and either displays the results to the user, or uses the results to perform some action.
5) OVAL Definition Repository:
Organizations publishing security advisories as OVAL Definitions and their products generating those definitions can be considered a repository of OVAL Definitions. This repository is made available to the community for further use.
Some products that have these OVAL capabilities are:
1) Hewlett-Packard Development Company, L.P: HP Server Automation (System Characteristic Producer):
HP’s Server Automation provides real-time vulnerability detection based on OVAL Vulnerability Definitions, automated patch management and internal compliance control thereby making: producing system characteristics, a key middle task. HP supports the OVAL standard to assist IT organizations in reducing security risks through automated vulnerability disclosure.
2) McAfee, Inc: McAfee Policy Auditor (Authoring Tool):
McAfee Policy Auditor is widely used as an Authoring tool. It automates the process required for system compliance audits. An organization may deploy a centralized audit validation system using McAfee’s Auditor. When a new system joins the network, it is immediately scanned by the organization’s vulnerability management, patch management, and configuration management tools based on the most up-to-date security advisories, patches, and policies.
3) Red Hat, Inc: Red Hat Security Advisories (Definition Repository):
Red Hat was a founding board member of the OVAL project and has been publishing OVAL Vulnerability Definitions for Red Hat Enterprise Linux Security Advisories since 2006.
4) U.S. Army CERDEC: Armadillo (Definition Evaluator):
Armadillo developed by U.S. Army CERDEC, is a configuration, vulnerability, patch, and software inventory network scanner. It identifies misconfigurations by consuming SCAP (Security Content Automation Protocol) bundles which contain XCCDF (Extensible Configuration Checklist Description Format) benchmarks (analogous to STIGs) which define the policy guidance and OVAL Definition files which define probes and expected values (analogous to SRRs).
Conclusion
OVAL is extremely useful. It provides better evaluation of security of the systems in an organization and easier information sharing. OVAL uses CVE and security measures recommended by top organizations for writing definitions, thus providing a high standard of security testing for systems. In addition, security administrators can modify, add or delete definitions based on their need. Many organizations run highly customized softwares for which they can develop personalized security checks.
The OVAL community is actively working to add more definitions and better the interpreter and fix bugs. OVAL has already generated results. OVAL language and vulnerability definitions are freely available for Linux, Solaris and Windows. Also, the OVAL Interpreter is available for Red Hat Linux, Linux and Windows distributions.
References:
1)http://oval.mitre.org/index.html
2)https://oval.mitre.org/language/about/structure.html
3)https://oval.mitre.org/language/about/overview.html
4)https://nvd.nist.gov/scap/docs/conference%20presentations/workshops/OVAL%20Tutorial%202%20-%20%20Definitions.pdf
5)https://nvd.nist.gov/scap/docs/conference%20presentations/workshops/OVAL%20Tutorial%201%20-%20Overview.pdf
6)https://oval.mitre.org/about/faqs.html
7)https://github.com/OVALProject/Language
8)https://www.scmagazine.com/hot-or-not-open-vulnerability-assessment-language/article/553575/
9)http://searchsecurity.techtarget.com/opinion/Vulnerability-testing-with-Open-Vulnerability-Assessment-Language
10)http://ovaldb.altx-soft.ru/Definitions.aspx
11)http://ovalproject.github.io/
12)https://oval.cisecurity.org/community/discuss
13)https://sourceforge.net/projects/ovaldi/
14)http://oval.mitre.org/adoption/
15)http://oval.mitre.org/adoption/requirements.html
16)https://kc.mcafee.com/resources/sites/MCAFEE/content/live/PRODUCT_DOCUMENTATION/24000/PD24693/en_US/pa_620_product_guide_en-us.pdf
17)https://www.redhat.com/security/data/oval/
18)https://oval.mitre.org/about/images/how_oval_works.pdf
