Skip to content

Commit 10f250b

Browse files
rr404AlteredCoderjdv
authored
Adding csv output creating report and details csv files (#16)
* adding csv output creating report and details csv files *proper csv format split from file output system + some facorization * factorized and reordered for more logical reding of the code * updated readme with format and saving instructions --------- Co-authored-by: Kevin KADOSH <kevin@crowdsec.net> Co-authored-by: jdv <julien@crowdsec.net>
1 parent ba80f87 commit 10f250b

10 files changed

Lines changed: 1081 additions & 210 deletions

File tree

‎README.md‎

Lines changed: 66 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -15,36 +15,41 @@ Your ultimate IP dex!
1515

1616
## Table of Contents
1717

18-
- [Introduction](#introduction)
19-
- [Prerequisites](#prerequisites)
20-
- [Quickstart](#quickstart)
21-
- [Install](#1-install)
22-
- [Make sure the binary is in your PATH](#2-make-sure-the-binary-is-in-your-path)
23-
- [Initialize the tool](#3-initialize-the-tool)
24-
- [Query an IP](#4-query-an-ip)
25-
- [Scan a file](#5-scan-a-file)
26-
- [Configuration](#configuration)
27-
- [User Guide](#user-guide)
28-
- [Scan an IP](#scan-an-ip)
29-
- [Refresh an IP](#refresh-an-ip)
30-
- [Scan a file](#scan-a-file-1)
31-
- [Refresh a file](#refresh-a-file)
32-
- [Display all reports](#display-all-reports)
33-
- [Showing a specific report](#showing-a-specific-report)
34-
- [Commands](#commands)
35-
- [`init`](#init)
36-
- [`report`](#report)
37-
- [List reports](#list-reports)
38-
- [View a report](#view-a-report)
39-
- [Delete a report](#delete-a-report)
40-
- [`search`](#search)
41-
- [Search IPs reported for a specific CVE](#search-ips-reported-for-a-specific-cve)
42-
- [Search IPs reported for HTTP scan since 30 minutes](#search-ips-reported-for-http-scan-since-30-minutes)
43-
- [Search malicious VPN or Proxy IPs since 1h and show all IPs](#search-malicious-vpn-or-proxy-ips-since-1h-and-show-all-ips)
44-
- [`config`](#config)
45-
- [Show config](#show-config)
46-
- [Set a new API Key](#set-a-new-api-key)
47-
- [License](#license)
18+
- [ipdex](#ipdex)
19+
- [Table of Contents](#table-of-contents)
20+
- [Introduction](#introduction)
21+
- [Prerequisites](#prerequisites)
22+
- [Quickstart](#quickstart)
23+
- [1. Install](#1-install)
24+
- [Install with Go](#install-with-go)
25+
- [macOS / Linux](#macos--linux)
26+
- [Linux](#linux)
27+
- [macOS](#macos)
28+
- [Windows](#windows)
29+
- [2. Make sure the binary is in your PATH](#2-make-sure-the-binary-is-in-your-path)
30+
- [3. Initialize the tool](#3-initialize-the-tool)
31+
- [4. Query an IP](#4-query-an-ip)
32+
- [5. Scan a file](#5-scan-a-file)
33+
- [Configuration](#configuration)
34+
- [User Guide](#user-guide)
35+
- [Scan an IP](#scan-an-ip)
36+
- [Refresh an IP](#refresh-an-ip)
37+
- [Scan a file](#scan-a-file)
38+
- [Refresh a file](#refresh-a-file)
39+
- [Output formats](#output-formats)
40+
- [Saving reports to files](#saving-reports-to-files)
41+
- [`report`](#report)
42+
- [List reports](#list-reports)
43+
- [View a report](#view-a-report)
44+
- [Delete a report](#delete-a-report)
45+
- [`search`](#search)
46+
- [Search IPs reported for a specific CVE](#search-ips-reported-for-a-specific-cve)
47+
- [Search IPs reported for HTTP scan since 30 minutes](#search-ips-reported-for-http-scan-since-30-minutes)
48+
- [Search malicious VPN or Proxy IPs since 1h and show all IPs](#search-malicious-vpn-or-proxy-ips-since-1h-and-show-all-ips)
49+
- [`config`](#config)
50+
- [Show config](#show-config)
51+
- [Set a new API Key](#set-a-new-api-key)
52+
- [License](#license)
4853

4954
---
5055

@@ -162,6 +167,8 @@ ipdex /var/log/nginx.log
162167

163168
<p align="center"> <img src="img/ipdex_file.svg" alt="ipdex scanning a file" width="900" /> </p>
164169

170+
**💡 Tip:** You can output results in different formats (`-o json`, `-o csv`) and save them to files using `--output-path`. See [Output formats](#output-formats) and [Saving reports to files](#saving-reports-to-files) for more details.
171+
165172
---
166173

167174
## Configuration
@@ -202,6 +209,31 @@ When running ipdex on a file that has been previously scanned, it will update th
202209
ipdex <filepath> -r
203210
```
204211

212+
### Output formats
213+
214+
ipdex supports multiple output formats to suit different use cases using the -o option:
215+
216+
- **human** (default): Interactive, colorized output optimized for terminal viewing
217+
- **json**: `-o json` Machine-readable JSON format for programmatic processing
218+
- **csv**: `-o csv` Comma-separated values format for spreadsheet analysis
219+
220+
### Saving reports to files
221+
222+
You can save reports to disk using the `--output-path` flag. This works with all output formats and automatically creates separate files for the report summary and the detailed IP information (if you used the -d option).
223+
224+
```bash
225+
# Save report as CSV files report and details
226+
ipdex ips.txt -o csv -d --output-path /path/to/output
227+
228+
# This creates:
229+
# - /path/to/output/report_<id>.csv (summary statistics)
230+
# - /path/to/output/report_<id>_details.csv (detailed IP information, when using -d flag)
231+
232+
# You can also do it for an existing report
233+
ipdex report show 18 -o csv --output-path /path/to/output -d
234+
235+
**Note:** When using `--output-path`, reports are saved to files in addition to being displayed in the terminal.
236+
205237
### Display all reports
206238

207239
```
@@ -242,7 +274,11 @@ ipdex report list
242274
#### View a report
243275

244276
```bash
277+
# View a report in human-readable format
245278
ipdex report show 2
279+
280+
# View report with details as CSV and save
281+
ipdex report show 2 -o csv --output-path ./exports -d
246282
```
247283

248284
#### Delete a report

‎cmd/ipdex/config/global.go‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
package config
22

33
var (
4-
OutputFormat string
5-
ForceRefresh bool
6-
Yes bool
7-
Detailed bool
8-
ReportName string
9-
Batching bool
4+
OutputFormat string
5+
OutputFilePath string
6+
ForceRefresh bool
7+
Yes bool
8+
Detailed bool
9+
ReportName string
10+
Batching bool
1011
)

‎cmd/ipdex/config/options.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ func GetConfigFolder() (string, error) {
3838

3939
func IsSupportedOutputFormat(outputFormat string) bool {
4040
switch outputFormat {
41-
case display.JSONFormat, display.HumanFormat:
41+
case display.JSONFormat, display.HumanFormat, display.CSVFormat:
4242
return true
4343
default:
4444
return false

‎cmd/ipdex/file/file.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,7 @@ func FileCommand(file string, forceRefresh bool, yes bool) {
206206
}
207207
}
208208
stats := reportClient.GetStats(report)
209-
if err := reportClient.Display(report, stats, viper.GetString(config.OutputFormatOption), config.Detailed); err != nil {
209+
if err := reportClient.Display(report, stats, viper.GetString(config.OutputFormatOption), config.Detailed, config.OutputFilePath); err != nil {
210210
style.Fatal(err.Error())
211211
}
212212
if !reportExist && outputFormat == display.HumanFormat {

‎cmd/ipdex/main.go‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,8 @@ func init() {
7575
rootCmd.Flags().BoolVarP(&config.ForceRefresh, "refresh", "r", false, "Force refresh an IP or all the IPs of a report")
7676
rootCmd.Flags().BoolVarP(&config.Yes, "yes", "y", false, "Say automatically yes to the warning about the number of IPs to scan")
7777
rootCmd.PersistentFlags().BoolVarP(&config.Detailed, "detailed", "d", false, "Show all informations about an IP or a report")
78-
rootCmd.PersistentFlags().StringVarP(&config.OutputFormat, "output", "o", "", "Output format: human or json")
78+
rootCmd.PersistentFlags().StringVarP(&config.OutputFormat, "output", "o", "", "Output format: human, json, or csv")
79+
rootCmd.PersistentFlags().StringVar(&config.OutputFilePath, "output-path", "", "Output file path for saving reports in the format specified by -o (saves report and details files separately)")
7980
rootCmd.Flags().StringVarP(&config.ReportName, "name", "n", "", "Report name when scanning a file or making a search query")
8081
rootCmd.Flags().BoolVarP(&config.Batching, "batch", "b", false, "Use batching to request the CrowdSec API. Make sure you have a premium API key to use this feature.")
8182
}

‎cmd/ipdex/report/show.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ func NewShowCommand() *cobra.Command {
5454
} else {
5555
style.Fatal("Please provide a report ID or file used in the report you want to show with `ipdex report show 1`")
5656
}
57-
if err := reportClient.Display(report, report.Stats, viper.GetString(config.OutputFormatOption), config.Detailed); err != nil {
57+
if err := reportClient.Display(report, report.Stats, viper.GetString(config.OutputFormatOption), config.Detailed, config.OutputFilePath); err != nil {
5858
style.Fatal(err.Error())
5959
}
6060
fmt.Println()

‎cmd/ipdex/search/search.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ func SearchCommand(query string, since string, maxResult int) {
118118
style.Fatalf("unable to create report: %s", err)
119119
}
120120
stats := reportClient.GetStats(report)
121-
if err := reportClient.Display(report, stats, viper.GetString(config.OutputFormatOption), config.Detailed); err != nil {
121+
if err := reportClient.Display(report, stats, viper.GetString(config.OutputFormatOption), config.Detailed, config.OutputFilePath); err != nil {
122122
style.Fatal(err.Error())
123123
}
124124
if outputFormat == display.HumanFormat {

0 commit comments

Comments
 (0)