Skip to content

Latest commit

 

History

History
35 lines (25 loc) · 1.78 KB

File metadata and controls

35 lines (25 loc) · 1.78 KB

Importer Overview

Importers are responsible for scraping vulnerability data such as vulnerabilities and their fixes and for storing the scraped information in a structured fashion. The structured data created by the importer then provides input to an improver (see :ref:`improver-overview`), which is responsible for creating a relational model for vulnerabilities, affected packages and fixed packages.

All importer implementation-related code is defined in :file:`vulnerabilities/importer.py`.

In addition, the framework-related code for actually invoking and processing the importers is located in :file:`vulnerabilities/import_runner.py`.

The importers, after scraping, provide AdvisoryData objects. These objects are then processed and inserted into the Advisory model.

While implementing an importer, it is important to make sure that the importer does not alter the upstream data at all. Its only job is to convert the data from a data source into structured -- yet non-relational -- data. This ensures that we always have a true copy of an advisory without any modifications.

Given that a lot of advisories publish version ranges of affected packages, it is necessary to store those ranges in a structured manner. Vers was designed to solve this problem. It has been implemented in the univers library whose development goes hand in hand with VulnerableCode.

The data imported by importers is not useful by itself: it must be processed into a relational model. The version ranges are required to be resolved into concrete ranges. These are achieved by Improvers (see :ref:`improver-overview` for details).

As of now, the following importers have been implemented in VulnerableCode: