Merge pull request GreedyBear-Project#1271 from GreedyBear-Project/develop

3.4.0

Author: regulartim

.github/.pre-commit-config.yaml

@@ -1,7 +1,7 @@
 repos:
 # Python linting with Ruff
 -   repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.12.7
+    rev: v0.15.11
     hooks:
       - id: ruff
         name: ruff-lint

.github/CONTRIBUTING.md

@@ -1,3 +1 @@
-GreedyBear is handled by the same maintainers of [IntelOwl](https://github.com/intelowlproject/IntelOwl/).
-
-So, please refer to the [Contribute guide](https://github.com/GreedyBear-Project/GreedyBear/wiki/Contribute)
+Please refer to our [Contribution guidelines](https://github.com/GreedyBear-Project/GreedyBear/wiki/Contribute).

README.md

@@ -1,48 +1,25 @@
 <p align="center"><img src="static/greedybear.png" width=350 height=404 alt="GreedyBear"/></p>
 
 # GreedyBear
-[![GitHub release (latest by date)](https://img.shields.io/github/v/release/intelowlproject/Greedybear)](https://github.com/intelowlproject/Greedybear/releases)
-[![GitHub Repo stars](https://img.shields.io/github/stars/intelowlproject/Greedybear?style=social)](https://github.com/intelowlproject/Greedybear/stargazers)
-[![Twitter Follow](https://img.shields.io/twitter/follow/intel_owl?style=social)](https://twitter.com/intel_owl)
-[![Linkedin](https://img.shields.io/badge/LinkedIn-0077B5?style=flat&logo=linkedin&logoColor=white)](https://www.linkedin.com/company/intelowl/)
+[![GitHub release (latest by date)](https://img.shields.io/github/v/release/GreedyBear-Project/Greedybear)](https://github.com/GreedyBear-Project/Greedybear/releases)
+[![GitHub Repo stars](https://img.shields.io/github/stars/GreedyBear-Project/Greedybear?style=social)](https://github.com/GreedyBear-Project/Greedybear/stargazers)
+![GitHub License](https://img.shields.io/github/license/GreedyBear-Project/GreedyBear)
 
+[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
-[![CodeQL](https://github.com/intelowlproject/GreedyBear/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/intelowlproject/GreedyBear/actions/workflows/codeql-analysis.yml)
-[![Dependency Review](https://github.com/intelowlproject/GreedyBear/actions/workflows/dependency_review.yml/badge.svg)](https://github.com/intelowlproject/GreedyBear/actions/workflows/dependency_review.yml)
-[![Pull request automation](https://github.com/intelowlproject/GreedyBear/actions/workflows/pull_request_automation.yml/badge.svg)](https://github.com/intelowlproject/GreedyBear/actions/workflows/pull_request_automation.yml)
+[![CodeQL](https://github.com/GreedyBear-Project/GreedyBear/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/GreedyBear-Project/GreedyBear/actions/workflows/codeql-analysis.yml)
+[![Dependency Review](https://github.com/GreedyBear-Project/GreedyBear/actions/workflows/dependency_review.yml/badge.svg)](https://github.com/GreedyBear-Project/GreedyBear/actions/workflows/dependency_review.yml)
+[![Pull request automation](https://github.com/GreedyBear-Project/GreedyBear/actions/workflows/pull_request_automation.yml/badge.svg)](https://github.com/GreedyBear-Project/GreedyBear/actions/workflows/pull_request_automation.yml)
 
-The project goal is to extract data of the attacks detected by a [TPOT](https://github.com/telekom-security/tpotce) or a cluster of them and to generate some feeds that can be used to prevent and detect attacks.
+The project goal is to extract attack data detected by a [T-Pot](https://github.com/telekom-security/tpotce) or a cluster of them and to generate some feeds that can be used to prevent and detect attacks. You can read the [official announcement here](https://www.honeynet.org/2021/12/27/new-project-available-greedybear/).
 
-[Official announcement here](https://www.honeynet.org/2021/12/27/new-project-available-greedybear/).
-
-## Documentation
-
-Documentation about GreedyBear installation, usage, configuration and contribution can be found at [this link](https://github.com/GreedyBear-Project/GreedyBear/wiki)
-
-## Public feeds
-
-There are public feeds provided by [The Honeynet Project](https://www.honeynet.org) in this [site](https://greedybear.honeynet.org). [Example](https://greedybear.honeynet.org/api/feeds/cowrie/all/recent.txt)
-
-Please do not perform too many requests to extract feeds or you will be banned.
-
-If you want to be updated regularly, please download the feeds only once every 10 minutes (this is the time between each internal update).
-
-To check all the available feeds, Please refer to our [usage guide](https://github.com/GreedyBear-Project/GreedyBear/wiki/Usage)
-
-
-## Enrichment Service
-
-GreedyBear provides an easy-to-query API to get the information available in GB regarding the queried observable (domain or IP address).
-
-To understand more, Please refer to our [usage guide](https://github.com/GreedyBear-Project/GreedyBear/wiki/Usage)
-
-## Run Greedybear on your environment
-The tool has been created not only to provide the feeds from The Honeynet Project's cluster of TPOTs.
-
-If you manage one or more T-POTs of your own, you can get the code of this application and run Greedybear on your environment.
-In this way, you are able to provide new feeds of your own.
-
-To install it locally, Please refer to our [installation guide](https://github.com/GreedyBear-Project/GreedyBear/wiki/Installation)
+## How to ...
+- **... try it out**: visit the [public instance](https://greedybear.honeynet.org) provided by [The Honeynet Project](https://www.honeynet.org) and take a look at a [threat intelligence live feed example](https://greedybear.honeynet.org/api/feeds/cowrie/all/recent.txt)
+- **... dive in**: read through our documentation in the [Wiki](https://github.com/GreedyBear-Project/GreedyBear/wiki) and explore GreedyBear's features
+- **... run your own instance**: to leverage everything GreedyBear has to offer, you might want to [install](https://github.com/GreedyBear-Project/GreedyBear/wiki/Installation) it and connect it to your own T-Pot
+- **... stay up to date**: [read](https://greedybear-project.github.io/) and [subscribe](https://greedybear-project.github.io/feed.xml) to our blog, where we regularly write about the most recent changes and new features
+- **... contact us**: using a Github [issue](https://github.com/GreedyBear-Project/GreedyBear/issues) or start a [discussion](https://github.com/GreedyBear-Project/GreedyBear/discussions)
+- **... contribute**: read through our [contribution guidelines](https://github.com/GreedyBear-Project/GreedyBear/wiki/Contribute), open an [issue](https://github.com/GreedyBear-Project/GreedyBear/issues), get assigned and raise a [pull request](https://github.com/GreedyBear-Project/GreedyBear/pulls)
 
 ## Sponsors and Acknowledgements
 
@@ -57,15 +34,23 @@ Thanks to [The Honeynet Project](https://www.honeynet.org) we are providing free
 #### Google Summer of Code
 <a href="https://summerofcode.withgoogle.com/"> <img style="border: 0.2px solid black" width=150 height=89 src="static/gsoc_logo.png" alt="GSoC logo"> </a>
 
-In 2026 we started participating to the [Google Summer of Code](https://summerofcode.withgoogle.com/) (GSoC)!
+In 2026 we started participating in the [Google Summer of Code](https://summerofcode.withgoogle.com/) (GSoC)!
 
 If you are interested in participating in the next Google Summer of Code, check all the info available in the [dedicated repository](https://github.com/intelowlproject/gsoc)!
 
-## Maintainers and Key Contributors
+## Maintainers and Contributors
 
 This project was started as a personal Christmas project by [Matteo Lodi](https://twitter.com/matte_lodi) in 2021.
 
 Special thanks to:
-* [Tim Leonhard](https://github.com/regulartim) for having greatly improved the project and added Machine Learning Models during his master thesis. He's the actual Principal Mantainer.
-* [Martina Carella](https://github.com/carellamartina) for having created the GUI during her master thesis.
-* [Daniele Rosetti](https://github.com/drosetti) for helping maintaining the Frontend.
+- [Tim Leonhard](https://github.com/regulartim) for having greatly improved the project and added Machine Learning Models during his master thesis. He's the current Principal Maintainer.
+- [Martina Carella](https://github.com/carellamartina) for having created the GUI during her master thesis.
+- [Daniele Rosetti](https://github.com/drosetti) for helping maintaining the Frontend.
+- and everyone who has contributed to GreedyBear!
+
+<a href="https://github.com/GreedyBear-Project/GreedyBear/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=GreedyBear-Project/GreedyBear" alt="GreedyBear contributors" />
+</a>
+
+## License
+Distributed under the MIT license. See [`LICENSE`](LICENSE) for the full text.

api/serializers.py

@@ -237,6 +237,7 @@ class FeedsResponseSerializer(serializers.Serializer):
     attacker_country = serializers.CharField(allow_null=True, allow_blank=True, max_length=120)
     attacker_country_code = serializers.CharField(allow_null=True, allow_blank=True, max_length=2)
     tags = TagSerializer(many=True, required=False, default=list)
+    sensors = SensorSerializer(many=True, required=False, default=list)
 
     def validate_feed_type(self, feed_type):
         logger.debug(f"FeedsResponseSerializer - validation feed_type: '{feed_type}'")

api/urls.py

@@ -15,6 +15,7 @@
     feeds_pagination,
     feeds_revoke,
     feeds_share,
+    feeds_tokens,
     general_honeypot_list,
     health_view,
     news_view,
@@ -30,6 +31,7 @@
     path("feeds/share", feeds_share),
     path("feeds/consume/<str:token>", feeds_consume),
     path("feeds/revoke/<str:token>", feeds_revoke),
+    path("feeds/tokens/", feeds_tokens),
     path("feeds/advanced/", feeds_advanced),
     path("feeds/asn/", feeds_asn),
     path("feeds/<str:feed_type>/<str:attack_type>/<str:prioritize>.<str:format_>", feeds),

api/views/cowrie_session.py

@@ -101,7 +101,7 @@ def cowrie_session_view(request):
     if include_similar:
         commands = {s.commands for s in sessions if s.commands}
         clusters = {cmd.cluster for cmd in commands if cmd.cluster is not None}
-        related_sessions = CowrieSession.objects.filter(commands__cluster__in=clusters).prefetch_related("source", "commands", "credentials")
+        related_sessions = CowrieSession.objects.filter(commands__cluster__in=clusters, duration__gt=0).prefetch_related("source", "commands", "credentials")
         sessions = sessions.union(related_sessions)
 
     response_data = {

api/views/feeds.py

@@ -41,6 +41,14 @@
     "prioritize",
 ]
 
+_TOKEN_LIST_FIELDS = (
+    "token_hash",
+    "reason",
+    "created_at",
+    "revoked",
+    "revoked_at",
+)
+
 
 @api_view([GET])
 @throttle_classes([FeedsThrottle])
@@ -145,13 +153,14 @@ def feeds_advanced(request):
         valid_feed_types,
         tag_key=request.query_params.get("tag_key", "").strip(),
         tag_value=request.query_params.get("tag_value", "").strip(),
+        include_sensors=True,
     )
     if paginate:
         paginator = CustomPageNumberPagination()
         iocs = paginator.paginate_queryset(iocs_queryset, request)
-        resp_data = feeds_response(request, iocs, feed_params, valid_feed_types, dict_only=True, verbose=verbose)
+        resp_data = feeds_response(request, iocs, feed_params, valid_feed_types, dict_only=True, verbose=verbose, include_sensors=True)
         return paginator.get_paginated_response(resp_data)
-    return feeds_response(request, iocs_queryset, feed_params, valid_feed_types, verbose=verbose)
+    return feeds_response(request, iocs_queryset, feed_params, valid_feed_types, verbose=verbose, include_sensors=True)
 
 
 @api_view(["GET"])
@@ -219,20 +228,27 @@ def feeds_share(request):
         port (int): Filter by destination port.
         start_date (str): Filter by start date (YYYY-MM-DD).
         end_date (str): Filter by end date (YYYY-MM-DD).
+        reason (str): Optional human-readable label for this share token (max 256 chars).
 
     Returns:
         Response: A JSON object containing the signed shareable URL.
     """
-    logger.info(f"request /api/feeds/share with params: {request.query_params}")
+    safe_params = {k: v for k, v in request.query_params.items() if k != "reason"}
+    logger.info(f"request /api/feeds/share with params: {safe_params}")
     feed_params = FeedRequestParams(request.query_params)
     data = vars(feed_params)
     # Remove internal or non-serializable objects if any
     data.pop("feed_type_sorting", None)
 
+    reason = request.query_params.get("reason", "").strip()[:256]
+
     # Generate signed token and persist a ShareToken record
     token = signing.dumps(data, salt="greedybear-feeds")
     token_hash = hashlib.sha256(token.encode()).hexdigest()
-    ShareToken.objects.get_or_create(token_hash=token_hash, defaults={"user": request.user})
+    ShareToken.objects.get_or_create(
+        token_hash=token_hash,
+        defaults={"user": request.user, "reason": reason},
+    )
 
     host = request.build_absolute_uri("/")
     share_url = f"{host}api/feeds/consume/{token}"
@@ -323,3 +339,32 @@ def feeds_revoke(request, token):
     share_token.revoked_at = timezone.now()
     share_token.save(update_fields=["revoked", "revoked_at"])
     return Response({"detail": "Token revoked successfully."}, status=status.HTTP_200_OK)
+
+
+@api_view([GET])
+@authentication_classes([CookieTokenAuthentication])
+@permission_classes([IsAuthenticated])
+def feeds_tokens(request):
+    """
+    List the calling user's share tokens with safe metadata.
+
+    Returns only non-sensitive fields: a truncated hash prefix (first 12 hex
+    chars), the reason label, creation timestamp, and revocation status.
+    The raw token is never stored and therefore cannot be returned.
+
+    Returns:
+        Response: A JSON list of token metadata objects.
+    """
+    logger.info("request /api/feeds/tokens/")
+    tokens = ShareToken.objects.filter(user=request.user).order_by("-created_at").values(*_TOKEN_LIST_FIELDS)
+    results = [
+        {
+            "hash_prefix": t["token_hash"][:12],
+            "reason": t["reason"],
+            "created_at": t["created_at"],
+            "revoked": t["revoked"],
+            "revoked_at": t["revoked_at"],
+        }
+        for t in tokens
+    ]
+    return Response(results)

api/views/statistics.py

@@ -86,18 +86,25 @@ def countries(self, request):
             request: The incoming request object.
 
         Returns:
-            Response: A JSON list of {country, count} objects ordered by count descending.
+            Response: A JSON list of {country, code, count} objects ordered by count descending.
         """
         delta, _ = self.__parse_range(self.request)
         qs = (
             IOC.objects.filter(last_seen__gte=delta)
             .exclude(attacker_country="")
             .filter(honeypots__active=True)
-            .values("attacker_country")
+            .values("attacker_country", "attacker_country_code")
             .annotate(count=Count("id", distinct=True))
             .order_by("-count")
         )
-        data = [{"country": item["attacker_country"], "count": item["count"]} for item in qs]
+        data = [
+            {
+                "country": item["attacker_country"],
+                "code": item["attacker_country_code"],
+                "count": item["count"],
+            }
+            for item in qs
+        ]
         return Response(data)
 
     @action(detail=False, methods=["get"])

api/views/utils.py

@@ -153,7 +153,9 @@ def get_valid_feed_types() -> frozenset[str]:
     return frozenset(feed_types)
 
 
-def get_queryset(request, feed_params, valid_feed_types, is_aggregated=False, serializer_class=FeedsRequestSerializer, tag_key="", tag_value=""):
+def get_queryset(
+    request, feed_params, valid_feed_types, is_aggregated=False, serializer_class=FeedsRequestSerializer, tag_key="", tag_value="", include_sensors=False
+):
     """
     Build a queryset to filter IOC data based on the request parameters.
 
@@ -172,6 +174,8 @@ def get_queryset(request, feed_params, valid_feed_types, is_aggregated=False, se
             - Default: `FeedsRequestSerializer`.
         tag_key (str, optional): Filter IOCs by tag key. Only passed from feeds_advanced.
         tag_value (str, optional): Filter IOCs by tag value (case-insensitive substring). Only passed from feeds_advanced.
+        include_sensors (bool, optional): If True, annotates sensors_json for each IOC.
+            Only passed from authenticated views like feeds_advanced. Default: False.
 
     Returns:
         QuerySet: The filtered queryset of IOC data.
@@ -252,6 +256,15 @@ def get_queryset(request, feed_params, valid_feed_types, is_aggregated=False, se
                     distinct=True,
                 )
             )
+            if include_sensors:
+                iocs = iocs.annotate(
+                    sensors_json=ArrayAgg(
+                        JSONObject(address=F("sensors__address"), label=F("sensors__label")),
+                        filter=Q(sensors__isnull=False),
+                        default=Value([]),
+                        distinct=True,
+                    )
+                )
         iocs = iocs.order_by(feed_params.ordering)
         iocs = iocs[: int(feed_params.feed_size)]
 
@@ -276,7 +289,7 @@ def ioc_as_dict(ioc, fields: set) -> dict:
     return {k: v for k, v in ioc.__dict__.items() if k in fields}
 
 
-def feeds_response(request=None, iocs=None, feed_params=None, valid_feed_types=None, dict_only=False, verbose=False):
+def feeds_response(request=None, iocs=None, feed_params=None, valid_feed_types=None, dict_only=False, verbose=False, include_sensors=False):
     """
     Format the IOC data into the requested format (e.g., JSON, CSV, TXT).
 
@@ -339,13 +352,19 @@ def feeds_response(request=None, iocs=None, feed_params=None, valid_feed_types=N
             required_fields = base_fields + verbose_only_fields if verbose else base_fields
 
             # `tags_json` is annotated in get_queryset (only for JSON format) to avoid conflicting
-            # with the `tags` reverse FK on IOC.  When the queryset comes from a repository method
+            # with the `tags` reverse FK on IOC. When the queryset comes from a repository method
             # that does not annotate `tags_json` (e.g. the ML scoring path), exclude the field.
+            # `sensors_json` follows the same pattern and is only annotated for authenticated views.
             if isinstance(iocs, list):
                 has_tags_annotation = bool(iocs) and hasattr(iocs[0], "tags_json")
+                has_sensors_annotation = include_sensors and bool(iocs) and hasattr(iocs[0], "sensors_json")
             else:
                 has_tags_annotation = "tags_json" in getattr(iocs, "query", type("", (), {"annotations": {}})()).annotations
+                has_sensors_annotation = include_sensors and "sensors_json" in getattr(iocs, "query", type("", (), {"annotations": {}})()).annotations
+
             required_fields = tuple(("tags_json" if f == "tags" else f) for f in required_fields if f != "tags" or has_tags_annotation)
+            if has_sensors_annotation:
+                required_fields = required_fields + ("sensors_json",)
 
             iocs_iter: object
             if isinstance(iocs, list):
@@ -362,6 +381,7 @@ def feeds_response(request=None, iocs=None, feed_params=None, valid_feed_types=N
                     "destination_port_count": len(ioc.get("destination_ports", [])),
                     "asn": ioc.get("autonomous_system", ""),
                     "tags": ioc.pop("tags_json", []),
+                    **({"sensors": ioc.pop("sensors_json", [])} if has_sensors_annotation else {}),
                 }
 
                 if not verbose:
@@ -557,7 +577,7 @@ def get_greedybear_news() -> list[dict]:
         feed = feedparser.parse(response.content)
 
         filtered_entries = sorted(
-            [entry for entry in feed.entries if "greedybear" in entry.get("title", "").lower() and entry.get("published_parsed")],
+            [entry for entry in feed.entries if entry.get("published_parsed")],
             key=lambda e: e.published_parsed,
             reverse=True,
         )