You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<li><p>The <aclass="reference internal" href="reference/generated/skrub.SessionEncoder.html#skrub.SessionEncoder" title="skrub.SessionEncoder"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">SessionEncoder</span></code></a> is now available. This encoder adds a <codeclass="docutils literal notranslate"><spanclass="pre">session_id</span></code>
618
+
column, which groups together events that occur within the given session gap.
619
+
Additionally, it is possible to provide a <codeclass="docutils literal notranslate"><spanclass="pre">split_by</span></code> column or list of columns
620
+
(e.g., user ID or (user ID, user device)) to compute sessions for each grouping
621
+
value.
622
+
<aclass="reference external" href="https://github.com/skrub-data/skrub/pull/1930">#1930</a> by <aclass="reference external" href="https://github.com/rcap107">Riccardo Cappuzzo</a>.</p></li>
623
+
<li><p>A new synthetic dataset generator for timestamped data and session-based</p></li>
624
+
</ul>
625
+
<blockquote>
626
+
<div><p>operations has been added: <aclass="reference internal" href="reference/generated/skrub.datasets.make_retail_events.html#skrub.datasets.make_retail_events" title="skrub.datasets.make_retail_events"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">make_retail_events()</span></code></a>.
627
+
<aclass="reference external" href="https://github.com/skrub-data/skrub/pull/1930">#1930</a> by <aclass="reference external" href="https://github.com/rcap107">Riccardo Cappuzzo</a>.</p>
628
+
</div></blockquote>
629
+
<ulclass="simple">
617
630
<li><p>The <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">DropSimilar</span></code> transformer has been added, for removing columns in a
618
631
dataframe that present high correlation with other columns. <aclass="reference external" href="https://github.com/skrub-data/skrub/pull/2023">#2023</a> by
<li><p>A new dataframe generator, <codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">datasets.toy_cities()</span></code>, has been added for
734
+
<li><p>A new dataframe generator, <aclass="reference internal" href="reference/generated/skrub.datasets.toy_cities.html#skrub.datasets.toy_cities" title="skrub.datasets.toy_cities"><codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">datasets.toy_cities()</span></code></a>, has been added for
722
735
use cases on dataframes with variable sizes and variable correlation between
723
736
columns. <aclass="reference external" href="https://github.com/skrub-data/skrub/pull/2042">#2042</a> by <aclass="reference external" href="https://github.com/emassoulie">Eloi Massoulié</a>.</p></li>
724
737
<li><p>A new selector function, <codeclass="xref py py-func docutils literal notranslate"><spanclass="pre">selectors.drop()</span></code>, has been added to drop columns
"\n# Sessions in time-based data: Predicting user purchases with the SessionEncoder\n\n.. |SessionEncoder| replace:: :class:`~skrub.SessionEncoder`\n.. |make_retail_events| replace:: :func:`~skrub.datasets.make_retail_events`\n.. |tabular_pipeline| replace:: :func:`~skrub.tabular_pipeline`\n.. |TableVectorizer| replace:: :class:`~skrub.TableVectorizer`\n.. |DummyClassifier| replace:: :class:`~sklearn.dummy.DummyClassifier`\n.. |TimeSeriesSplit| replace:: :class:`~sklearn.model_selection.TimeSeriesSplit`\n.. |BaseEstimator| replace:: :class:`~sklearn.base.BaseEstimator`\n.. |TransformerMixin| replace:: :class:`~sklearn.base.TransformerMixin`\n\nThis example shows how to use |SessionEncoder| in a scikit-learn pipeline to\ncreate session-level features (sessionization) for conversion prediction, that is\npredicting whether a user session will eventually lead to a purchase.\n\n.. topic:: What is sessionization?\n\n Sessionization is the process of grouping a sequence of events (like user\n interactions) into meaningful sessions. A session typically starts fresh or\n after a period of inactivity. For example, in an online retail context, you\n might define a new session whenever more than 30 minutes pass with no activity\n from a user. This allows you to extract session-level features (like the total\n number of events in a session or the dominant device type used) which often have\n greater predictive power than raw individual events.\n\nWe will:\n\n1. Use |make_retail_events| to generate synthetic retail event data\n2. Build a baseline classifier on raw event-level features with the |tabular_pipeline|\n3. Add session-level and historical features with |SessionEncoder|\n4. Train the same model again and compare ROC-AUC\n\nThe data includes columns such as event type, device type, viewed price, and\ntimestamp. The target is binary: whether the session eventually contains a\npurchase event or not.\n"
8
+
]
9
+
},
10
+
{
11
+
"cell_type": "markdown",
12
+
"metadata": {},
13
+
"source": [
14
+
"Since this is temporal data, we use a time-aware CV strategy with\n|TimeSeriesSplit| to avoid leakage. We reuse the same splitter for all evaluations.\nThe dataset is sorted by timestamp, so the training set will always contain only\npast data relative to the test set.\n\n"
"The data contains 5000 events from 20 users, where each event is timestamped.\nOther columns include the event type, device used by the user, page category,\ntime spent on page and price of the item. The target variable indicates whether\na user session eventually contains a purchase event: all events in that session\nwill have a target value of 1 if a purchase happens, and 0 otherwise.\n\n"
51
+
]
52
+
},
53
+
{
54
+
"cell_type": "markdown",
55
+
"metadata": {},
56
+
"source": [
57
+
"## Sanity check: evaluate a DummyClassifier on raw event data\nWe begin by evaluating a |DummyClassifier| on the original event data\n(without session features). Since it's a |DummyClassifier|, we expect\nchance-level performance (ROC-AUC of 0.5).\n\n"
58
+
]
59
+
},
60
+
{
61
+
"cell_type": "code",
62
+
"execution_count": null,
63
+
"metadata": {
64
+
"collapsed": false
65
+
},
66
+
"outputs": [],
67
+
"source": [
68
+
"from sklearn.dummy import DummyClassifier\nfrom sklearn.model_selection import cross_val_score\n\ndummy = DummyClassifier(strategy=\"most_frequent\")\n\nscores = cross_val_score(dummy, X, y, cv=splitter, scoring=\"roc_auc\")\nprint(f\"ROC-AUC with DummyClassifier: {scores.mean():.3f}\")"
69
+
]
70
+
},
71
+
{
72
+
"cell_type": "markdown",
73
+
"metadata": {},
74
+
"source": [
75
+
"## First attempt: training a model without using session-level features\nWe first use the |tabular_pipeline| on raw event-level data, without any session\nencoding or aggregation. This serves as a baseline to compare against the enriched\nmodel later.\nRemember that the |tabular_pipeline| will automatically add a |TableVectorizer|\nto perform feature engineering, so the model can still learn from the raw event\nfeatures. However, it won't be able to directly capture session-level patterns.\n\n"
76
+
]
77
+
},
78
+
{
79
+
"cell_type": "code",
80
+
"execution_count": null,
81
+
"metadata": {
82
+
"collapsed": false
83
+
},
84
+
"outputs": [],
85
+
"source": [
86
+
"from skrub import tabular_pipeline\n\nmodel = tabular_pipeline(\"classification\")\n\nscores = cross_val_score(model, X, y, cv=splitter, scoring=\"roc_auc\")\nprint(f\"ROC-AUC without session encoding: {scores.mean():.3f}\")"
87
+
]
88
+
},
89
+
{
90
+
"cell_type": "markdown",
91
+
"metadata": {},
92
+
"source": [
93
+
"The model is not performing much better than the DummyClassifier, which suggests\nthat raw event-level features are not sufficient for good conversion prediction.\nThis baseline is limited because it cannot directly use session-level behavior\n(for example, whether \"add_to_cart\" happened in the same session).\n\n"
94
+
]
95
+
},
96
+
{
97
+
"cell_type": "markdown",
98
+
"metadata": {},
99
+
"source": [
100
+
"## A better approach: session encoding and aggregation\nNext, we use the |SessionEncoder| to create session-level features that we can\naggregate over. We define a session boundary as \"a user has been inactive for\nmore than 30 minutes\". The |SessionEncoder| will create a new column\n``timestamp_session_id`` that assigns a unique session ID to each session detected.\nThe parameter ``session_gap=30 * 60`` specifies the inactivity threshold in\nseconds (30 minutes).\n\nNote that session-based features involve aggregations, which must be performed\nonly on the training data within each fold to avoid leakage. In a scikit-learn\npipeline, we can achieve this by using |SessionEncoder| followed by a custom\ntransformer that computes session aggregates, and ensures that the pipeline is\nproperly fitted within each fold of cross-validation.\n\n"
101
+
]
102
+
},
103
+
{
104
+
"cell_type": "code",
105
+
"execution_count": null,
106
+
"metadata": {
107
+
"collapsed": false
108
+
},
109
+
"outputs": [],
110
+
"source": [
111
+
"from skrub import SessionEncoder, tabular_pipeline\n\nse = SessionEncoder(\"timestamp\", split_by=\"user_id\", session_gap=30 * 60)\n# Here we fit the SessionEncoder on the entire dataset for demonstration purposes\nX_sessions = se.fit_transform(X)\nX_sessions.head()"
112
+
]
113
+
},
114
+
{
115
+
"cell_type": "markdown",
116
+
"metadata": {},
117
+
"source": [
118
+
"## Defining a custom transformer for session-level aggregation\nTo avoid data leakage and maintain a clean pipeline, we can create a custom\ntransformer that inherits from |BaseEstimator| and |TransformerMixin| and\ncomputes session-level aggregates within a scikit-learn pipeline.\nThis transformer will be fitted and applied separately within each fold of\ncross-validation, ensuring that session features are computed only on the training\ndata of each fold.\n\n"
"Then, we create a pipeline that includes the |SessionEncoder|, our custom\n``SessionAggregator``, and the |tabular_pipeline| for classification. This\npipeline will be used in cross-validation to evaluate the model\nwith session features.\n\n"
137
+
]
138
+
},
139
+
{
140
+
"cell_type": "code",
141
+
"execution_count": null,
142
+
"metadata": {
143
+
"collapsed": false
144
+
},
145
+
"outputs": [],
146
+
"source": [
147
+
"from sklearn.pipeline import make_pipeline\n\nmodel = make_pipeline(se, SessionAggregator(), tabular_pipeline(\"classification\"))\nscores = cross_val_score(model, X, y, cv=splitter, scoring=\"roc_auc\")\nprint(\"ROC-AUC with session encoding:\", scores.mean())"
148
+
]
149
+
},
150
+
{
151
+
"cell_type": "markdown",
152
+
"metadata": {},
153
+
"source": [
154
+
"As expected the model with session encoding performs much better than the baseline\nwithout session features, demonstrating the value of sessionization for conversion\nprediction.\n\nThe fact that we are working with aggregation means that it was necessary to\ncreate a custom transformer to compute session-level features. However, this situation\ncan be avoided entirely by using the skrub DataOps workflow, which allows for more\nflexible data transformations without needing to fit everything within a\nscikit-learn pipeline.\n\n"
0 commit comments