Add event deduplication via id and timestamp params (#125)

* Add event deduplication via id and timestamp params

Change track() and track_anonymous() to accept a data dict instead of
**kwargs, with optional id and timestamp keyword arguments.

The id parameter accepts a ULID for event deduplication. The timestamp
parameter sets the event time (epoch seconds). Invalid timestamps are
silently dropped. Passing data=None sends an empty data dict.

This is a breaking change: callers must switch from keyword arguments
to a dict for event data attributes.

Refs #98

Author: hownowstephen

CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## Unreleased
+### Added
+- Add support for optional top-level `id` and `timestamp` event fields in `track()` and `track_anonymous()`.
+
+### Changed
+- `track()` and `track_anonymous()` now take custom event attributes in the `data` dict instead of arbitrary keyword arguments.
+
 ## [2.4]
 ### Added
 - Add support for sending transactional in-app messages [#113](https://github.com/customerio/customerio-python/pull/113)

README.md

@@ -28,7 +28,7 @@ from customerio import CustomerIO, Regions
 cio = CustomerIO(site_id, api_key, region=Regions.US)
 cio.identify(id="5", email='customer@example.com', name='Bob', plan='premium')
 cio.track(customer_id="5", name='purchased')
-cio.track(customer_id="5", name='purchased', price=23.45)
+cio.track(customer_id="5", name='purchased', data={"price": 23.45})
 ```
 
 ### Instantiating customer.io object
@@ -51,7 +51,7 @@ Only the id field is used to identify the customer here.  Using an existing id w
 a different email (or any other attribute) will update/overwrite any pre-existing
 values for that field.
 
-You can pass any keyword arguments to the `identify` and `track` methods. These kwargs will be converted to custom attributes.
+You can pass any keyword arguments to the `identify` method. These kwargs will be converted to custom attributes.
 
 See original REST documentation [here](http://customer.io/docs/api/track/#operation/identify)
 
@@ -64,13 +64,27 @@ cio.track(customer_id="5", name='purchased')
 ### Track a custom event with custom data values
 
 ```python
-cio.track(customer_id="5", name='purchased', price=23.45, product="widget")
+cio.track(customer_id="5", name='purchased', data={"price": 23.45, "product": "widget"})
 ```
 
-You can pass any keyword arguments to the `identify` and `track` methods. These kwargs will be converted to custom attributes.
+Pass custom event attributes to `track` in the `data` dict.
 
 See original REST documentation [here](http://customer.io/docs/api/track/#operation/track)
 
+### Track a custom event with an event id or timestamp
+
+```python
+cio.track(
+  customer_id="5",
+  name='purchased',
+  data={"price": 23.45, "product": "widget"},
+  id="01HB4HBDKTFWYZCK01DMRSWRFD",
+  timestamp=1561231234
+)
+```
+
+Pass `id` to provide a unique event identifier for deduplication. Pass `timestamp` to set the event time. These fields are sent as top-level event fields, not as custom attributes in `data`.
+
 ### Backfill a custom event
 
 ```python
@@ -92,24 +106,34 @@ cio.backfill(customer_id, event_type, event_timestamp, price=45.67)
 
 Event timestamp may be passed as a ```datetime.datetime``` object, an integer or a string UNIX timestamp
 
-Keyword arguments to backfill work the same as a call to ```cio.track```.
+Keyword arguments to backfill are converted to custom event attributes.
 
 See original REST documentation [here](http://customer.io/docs/api/track/#operation/track)
 
 ### Track an anonymous event
 
 ```python
-cio.track_anonymous(anonymous_id="anon-event", name="purchased", price=23.45, product="widget")
+cio.track_anonymous(
+  anonymous_id="anon-event",
+  name="purchased",
+  data={"price": 23.45, "product": "widget"}
+)
 ```
 
 An anonymous event is an event associated with a person you haven't identified. The event requires an `anonymous_id` representing the unknown person and an event `name`. When you identify a person, you can set their `anonymous_id` attribute. If [event merging](https://customer.io/docs/anonymous-events/#turn-on-merging) is turned on in your workspace, and the attribute matches the `anonymous_id` in one or more events that were logged within the last 30 days, we associate those events with the person.
 
+Like `track`, `track_anonymous` accepts custom event attributes in `data` and optional top-level `id` and `timestamp` fields.
+
 #### Anonymous invite events
 
 If you previously sent [invite events](https://customer.io/docs/journeys/anonymous-invite-emails/), you can achieve the same functionality by sending an anonymous event with the anonymous identifier set to `None`. To send anonymous invites, your event *must* include a `recipient` attribute. 
 
 ```python
-cio.track_anonymous(anonymous_id=None, name="invite", first_name="alex", recipient="alex.person@example.com")
+cio.track_anonymous(
+  anonymous_id=None,
+  name="invite",
+  data={"first_name": "alex", "recipient": "alex.person@example.com"}
+)
 ```
 
 ### Delete a customer profile
@@ -124,7 +148,7 @@ This method returns nothing.  Attempts to delete non-existent customers will not
 See original REST documentation [here](https://customer.io/docs/api/track/#operation/delete)
 
 
-You can pass any keyword arguments to the `identify` and `track` methods. These kwargs will be converted to custom attributes.
+You can pass any keyword arguments to the `identify` method. These kwargs will be converted to custom attributes.
 
 ### Merge duplicate customer profiles
 

customerio/track.py

@@ -91,24 +91,18 @@ def identify(self, id, **kwargs):
         url = self.get_customer_query_string(id)
         return self.send_request("PUT", url, kwargs)
 
-    def track(self, customer_id, name, **data):
+    def track(self, customer_id, name, data=None, id=None, timestamp=None):
         """Track an event for a given customer_id."""
         if not customer_id:
             raise CustomerIOException("customer_id cannot be blank in track")
         url = self.get_event_query_string(customer_id)
-        post_data = {
-            "name": name,
-            "data": self._sanitize(data),
-        }
+        post_data = self._build_event(name, data, id=id, timestamp=timestamp)
         return self.send_request("POST", url, post_data)
 
-    def track_anonymous(self, anonymous_id, name, **data):
+    def track_anonymous(self, anonymous_id, name, data=None, id=None, timestamp=None):
         """Track an event for a given anonymous_id."""
         url = self.get_events_query_string()
-        post_data = {
-            "name": name,
-            "data": self._sanitize(data),
-        }
+        post_data = self._build_event(name, data, id=id, timestamp=timestamp)
         if anonymous_id:
             post_data["anonymous_id"] = anonymous_id
 
@@ -149,6 +143,27 @@ def backfill(self, customer_id, name, timestamp, **data):
 
         return self.send_request("POST", url, post_data)
 
+    def _build_event(self, name, data=None, id=None, timestamp=None):
+        post_data = {
+            "name": name,
+            "data": self._sanitize(data or {}),
+        }
+        if id is not None:
+            post_data["id"] = id
+        if timestamp is not None:
+            if isinstance(timestamp, datetime):
+                timestamp = self._datetime_to_timestamp(timestamp)
+            elif isinstance(timestamp, int):
+                pass
+            else:
+                try:
+                    timestamp = int(timestamp)
+                except (ValueError, TypeError, OverflowError):
+                    timestamp = None
+            if timestamp is not None:
+                post_data["timestamp"] = timestamp
+        return post_data
+
     def delete(self, customer_id):
         """Delete a customer profile."""
         if not customer_id:

tests/test_customerio.py

@@ -17,6 +17,23 @@
 urllib3.disable_warnings()
 
 
+class TestCustomerIOTrackSignatures(unittest.TestCase):
+    def setUp(self):
+        self.cio = CustomerIO(site_id="siteid", api_key="apikey")
+
+    def test_track_rejects_event_data_keyword_arguments(self):
+        with self.assertRaises(TypeError):
+            self.cio.track(customer_id="5", name="purchased", price=23.45)
+
+    def test_track_anonymous_rejects_event_data_keyword_arguments(self):
+        with self.assertRaises(TypeError):
+            self.cio.track_anonymous(
+                anonymous_id=None,
+                name="invite",
+                recipient="alex.person@example.com",
+            )
+
+
 class TestCustomerIO(HTTPSTestCase):
     """Starts server which the client connects to in the following tests"""
 
@@ -131,11 +148,110 @@ def test_track_call(self):
             )
         )
 
-        self.cio.track(customer_id=1, name="sign_up", email="john@test.com")
+        self.cio.track(customer_id=1, name="sign_up", data={"email": "john@test.com"})
 
         with self.assertRaises(TypeError):
             self.cio.track(random_attr="some_value")
 
+    def test_track_with_id(self):
+        self.cio.http.hooks = dict(
+            response=partial(
+                self._check_request,
+                rq={
+                    "method": "POST",
+                    "url_suffix": "/customers/1/events",
+                    "body": {
+                        "name": "purchase",
+                        "data": {"type": "socks"},
+                        "id": "01HB4HBDKTFWYZCK01DMRSWRFD",
+                    },
+                },
+            )
+        )
+
+        self.cio.track(1, "purchase", {"type": "socks"}, id="01HB4HBDKTFWYZCK01DMRSWRFD")
+
+    def test_track_without_id(self):
+        self.cio.http.hooks = dict(
+            response=partial(
+                self._check_request,
+                rq={
+                    "method": "POST",
+                    "url_suffix": "/customers/1/events",
+                    "body": {"name": "purchase", "data": {"type": "socks"}},
+                },
+            )
+        )
+
+        self.cio.track(1, "purchase", {"type": "socks"})
+
+    def test_track_with_timestamp(self):
+        self.cio.http.hooks = dict(
+            response=partial(
+                self._check_request,
+                rq={
+                    "method": "POST",
+                    "url_suffix": "/customers/1/events",
+                    "body": {
+                        "name": "purchase",
+                        "data": {"type": "socks"},
+                        "timestamp": 1561231234,
+                    },
+                },
+            )
+        )
+
+        self.cio.track(1, "purchase", {"type": "socks"}, timestamp=1561231234)
+
+    def test_track_with_id_and_timestamp(self):
+        self.cio.http.hooks = dict(
+            response=partial(
+                self._check_request,
+                rq={
+                    "method": "POST",
+                    "url_suffix": "/customers/1/events",
+                    "body": {
+                        "name": "purchase",
+                        "data": {"type": "socks"},
+                        "id": "01HB4HBDKTFWYZCK01DMRSWRFD",
+                        "timestamp": 1561231234,
+                    },
+                },
+            )
+        )
+
+        self.cio.track(
+            1, "purchase", {"type": "socks"}, id="01HB4HBDKTFWYZCK01DMRSWRFD", timestamp=1561231234
+        )
+
+    def test_track_with_invalid_timestamp(self):
+        self.cio.http.hooks = dict(
+            response=partial(
+                self._check_request,
+                rq={
+                    "method": "POST",
+                    "url_suffix": "/customers/1/events",
+                    "body": {"name": "purchase", "data": {"type": "socks"}},
+                },
+            )
+        )
+
+        self.cio.track(1, "purchase", {"type": "socks"}, timestamp="not-a-timestamp")
+
+    def test_track_with_no_data(self):
+        self.cio.http.hooks = dict(
+            response=partial(
+                self._check_request,
+                rq={
+                    "method": "POST",
+                    "url_suffix": "/customers/1/events",
+                    "body": {"name": "login", "data": {}},
+                },
+            )
+        )
+
+        self.cio.track(1, "login")
+
     def test_track_anonymous_call(self):
         self.cio.http.hooks = dict(
             response=partial(
@@ -154,7 +270,71 @@ def test_track_anonymous_call(self):
             )
         )
 
-        self.cio.track_anonymous(anonymous_id=123, name="sign_up", email="john@test.com")
+        self.cio.track_anonymous(anonymous_id=123, name="sign_up", data={"email": "john@test.com"})
+
+    def test_track_anonymous_invite_with_data_dict(self):
+        self.cio.http.hooks = dict(
+            response=partial(
+                self._check_request,
+                rq={
+                    "method": "POST",
+                    "authorization": _basic_auth_str("siteid", "apikey"),
+                    "content_type": "application/json",
+                    "url_suffix": "/events",
+                    "body": {
+                        "data": {
+                            "first_name": "alex",
+                            "recipient": "alex.person@example.com",
+                        },
+                        "name": "invite",
+                    },
+                },
+            )
+        )
+
+        self.cio.track_anonymous(
+            anonymous_id=None,
+            name="invite",
+            data={"first_name": "alex", "recipient": "alex.person@example.com"},
+        )
+
+    def test_track_anonymous_with_id(self):
+        self.cio.http.hooks = dict(
+            response=partial(
+                self._check_request,
+                rq={
+                    "method": "POST",
+                    "url_suffix": "/events",
+                    "body": {
+                        "name": "purchase",
+                        "data": {},
+                        "anonymous_id": "anon-123",
+                        "id": "01HB4HBDKTFWYZCK01DMRSWRFD",
+                    },
+                },
+            )
+        )
+
+        self.cio.track_anonymous("anon-123", "purchase", id="01HB4HBDKTFWYZCK01DMRSWRFD")
+
+    def test_track_anonymous_with_timestamp(self):
+        self.cio.http.hooks = dict(
+            response=partial(
+                self._check_request,
+                rq={
+                    "method": "POST",
+                    "url_suffix": "/events",
+                    "body": {
+                        "name": "purchase",
+                        "data": {"type": "socks"},
+                        "anonymous_id": "anon-123",
+                        "timestamp": 1561231234,
+                    },
+                },
+            )
+        )
+
+        self.cio.track_anonymous("anon-123", "purchase", {"type": "socks"}, timestamp=1561231234)
 
     def test_pageview_call(self):
         self.cio.http.hooks = dict(
@@ -417,7 +597,7 @@ def test_ids_are_encoded_in_url(self):
                 },
             )
         )
-        self.cio.track(customer_id="1 ", name="test")
+        self.cio.track(customer_id="1 ", name="test", data={})
 
         self.cio.http.hooks = dict(
             response=partial(