Skip to content

Commit 90b3bd7

Browse files
asfernandesasfernandes
authored
Feature #9047 - Add GROUPS unit for window frames (#9048)
1 parent 7cfff87 commit 90b3bd7

8 files changed

Lines changed: 128 additions & 15 deletions

File tree

‎doc/sql.extensions/README.window_functions.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Syntax:
2323
ORDER BY <expr> [<direction>] [<nulls placement>] [, <expr> [<direction>] [<nulls placement>]] ...
2424
2525
<window frame> ::=
26-
{RANGE | ROWS} <window frame extent> [<window frame exclusion>]
26+
{RANGE | ROWS | GROUPS} <window frame extent> [<window frame exclusion>]
2727
2828
<window frame extent> ::=
2929
{<window frame start> | <window frame between>}
@@ -272,17 +272,19 @@ And the result set:
272272

273273
It's possible to specify the frame that some window functions work. The frame is divided in three piecies: unit, start bound and end bound.
274274

275-
The unit `RANGE` or `ROWS` defines how the bounds `<expr> PRECEDING`, `<expr> FOLLOWING` and `CURRENT ROW` works.
275+
The unit `RANGE`, `ROWS` or `GROUPS` defines how the bounds `<expr> PRECEDING`, `<expr> FOLLOWING` and `CURRENT ROW` works.
276276

277277
With `RANGE`, the `ORDER BY` should specify only one expression, and that expression should be of a numeric, date, time or timestamp type. For `<expr> PRECEDING` and `<expr> FOLLOWING` bounds, `<expr>` is respectively subtracted or added to the order expression, and for `CURRENT ROW` only the order expression is used. Then, all rows (inside the partition) between the bounds are considered part of the resulting window frame.
278278

279279
With `ROWS`, order expressions is not limited by number or types. In this case, `<expr> PRECEDING`, `<expr> FOLLOWING` and `CURRENT ROW` relates to the row position under the partition, and not to the order keys values.
280280

281-
`UNBOUNDED PRECEDING` and `UNBOUNDED FOLLOWING` work identically with `RANGE` and `ROWS`. `UNBOUNDED PRECEDING` looks for the first row and `UNBOUNDED FOLLOWING` the last one, always inside the partition.
281+
With `GROUPS`, order expressions are not limited by number or types. In this case, `<expr> PRECEDING`, `<expr> FOLLOWING` and `CURRENT ROW` relate to peer groups under the partition. Peers are rows with the same `ORDER BY` values. If there is no `ORDER BY`, all rows in the partition are peers and the partition has one group.
282+
283+
`UNBOUNDED PRECEDING` and `UNBOUNDED FOLLOWING` work identically with `RANGE`, `ROWS` and `GROUPS`. `UNBOUNDED PRECEDING` looks for the first row and `UNBOUNDED FOLLOWING` the last one, always inside the partition.
282284

283285
The frame syntax with `<window frame start>` specifies the start frame, with the end frame being `CURRENT ROW`.
284286

285-
The optional frame exclusion clause (FB 6.0) is part of the frame clause and removes rows from the frame after its bounds have been evaluated. It can only be specified together with an explicit `ROWS` or `RANGE` frame. `EXCLUDE NO OTHERS` is the default and keeps the frame unchanged.
287+
The optional frame exclusion clause (FB 6.0) is part of the frame clause and removes rows from the frame after its bounds have been evaluated. It can only be specified together with an explicit `ROWS`, `RANGE` or `GROUPS` frame. `EXCLUDE NO OTHERS` is the default and keeps the frame unchanged.
286288

287289
`EXCLUDE CURRENT ROW` removes only the current row from the frame.
288290

@@ -377,7 +379,7 @@ Some window functions discard frames and frame exclusions. `ROW_NUMBER`, `LAG` a
377379

378380
## 6. Named windows (FB 4.0)
379381

380-
To avoid write repetitive or confusing expressions, windows can be named in a query with the `WINDOW` clause. A named window can be used in `OVER` to reference a window definition and can also be used as a base window of another named or inline (`OVER`) window. A window with frame (`ROWS` or `RANGE` clauses) can't be used as base window (but can be used with `OVER <window name>`). And a window with a base window can't have `PARTITION BY` nor can override `ORDER BY` of a base window.
382+
To avoid write repetitive or confusing expressions, windows can be named in a query with the `WINDOW` clause. A named window can be used in `OVER` to reference a window definition and can also be used as a base window of another named or inline (`OVER`) window. A window with frame (`ROWS`, `RANGE` or `GROUPS` clauses) can't be used as base window (but can be used with `OVER <window name>`). And a window with a base window can't have `PARTITION BY` nor can override `ORDER BY` of a base window.
381383

382384
In a query with multiple `SELECT` and `WINDOW` clauses (for example, with subqueries), the window name scope is bound only to its query context, that is, a window name from an inner or outer context could not be used in another context. As such, the same window name definition could be used at different contexts.
383385

‎src/common/ParserTokens.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,6 +247,7 @@ PARSER_TOKEN(TOK_GRANT, "GRANT", false)
247247
PARSER_TOKEN(TOK_GRANTED, "GRANTED", true)
248248
PARSER_TOKEN(TOK_GREATEST, "GREATEST", false)
249249
PARSER_TOKEN(TOK_GROUP, "GROUP", false)
250+
PARSER_TOKEN(TOK_GROUPS, "GROUPS", false)
250251
PARSER_TOKEN(TOK_HASH, "HASH", true)
251252
PARSER_TOKEN(TOK_HAVING, "HAVING", false)
252253
PARSER_TOKEN(TOK_HEX_DECODE, "HEX_DECODE", true)

‎src/dsql/ExprNodes.h‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1492,8 +1492,8 @@ class WindowClause final : public DsqlNode<WindowClause, ExprNode::TYPE_WINDOW_C
14921492
{
14931493
// Warning: used in BLR
14941494
RANGE = 0,
1495-
ROWS
1496-
//// TODO: SQL-2013: GROUPS
1495+
ROWS,
1496+
GROUPS
14971497
};
14981498

14991499
public:

‎src/dsql/parse.y‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -713,6 +713,7 @@ using namespace Firebird;
713713
%token <metaNamePtr> FORMAT
714714
%token <metaNamePtr> GENERATE_SERIES
715715
%token <metaNamePtr> GREATEST
716+
%token <metaNamePtr> GROUPS
716717
%token <metaNamePtr> LEAST
717718
%token <metaNamePtr> LISTAGG
718719
%token <metaNamePtr> LTRIM
@@ -4886,6 +4887,7 @@ keyword_or_column
48864887
| CALL
48874888
| CURRENT_SCHEMA
48884889
| GREATEST
4890+
| GROUPS
48894891
| LEAST
48904892
| LISTAGG
48914893
| LTRIM
@@ -9003,6 +9005,10 @@ window_frame_extent
90039005
{ $$ = newNode<WindowClause::FrameExtent>(WindowClause::FrameExtent::Unit::RANGE); }
90049006
window_frame($2)
90059007
{ $$ = $2; }
9008+
| GROUPS
9009+
{ $$ = newNode<WindowClause::FrameExtent>(WindowClause::FrameExtent::Unit::GROUPS); }
9010+
window_frame($2)
9011+
{ $$ = $2; }
90069012
| ROWS
90079013
{ $$ = newNode<WindowClause::FrameExtent>(WindowClause::FrameExtent::Unit::ROWS); }
90089014
window_frame($2)

‎src/include/firebird/impl/msg/jrd.h‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -797,7 +797,7 @@ FB_IMPL_MSG(JRD, 795, no_cursor, -901, "07", "005", "Cannot open cursor for non-
797797
FB_IMPL_MSG(JRD, 796, dsql_window_incompat_frames, -104, "42", "000", "If <window frame bound 1> specifies @1, then <window frame bound 2> shall not specify @2")
798798
FB_IMPL_MSG(JRD, 797, dsql_window_range_multi_key, -104, "42", "000", "RANGE based window with <expr> {PRECEDING | FOLLOWING} cannot have ORDER BY with more than one value")
799799
FB_IMPL_MSG(JRD, 798, dsql_window_range_inv_key_type, -104, "42", "000", "RANGE based window with <offset> PRECEDING/FOLLOWING must have a single ORDER BY key of numerical, date, time or timestamp types")
800-
FB_IMPL_MSG(JRD, 799, dsql_window_frame_value_inv_type, -104, "42", "000", "Window RANGE/ROWS PRECEDING/FOLLOWING value must be of a numerical type")
800+
FB_IMPL_MSG(JRD, 799, dsql_window_frame_value_inv_type, -104, "42", "000", "Window RANGE/ROWS/GROUPS PRECEDING/FOLLOWING value must be of a numerical type")
801801
FB_IMPL_MSG(JRD, 800, window_frame_value_invalid, -833, "42", "000", "Invalid PRECEDING or FOLLOWING offset in window function: cannot be negative")
802802
FB_IMPL_MSG(JRD, 801, dsql_window_not_found, -833, "42", "000", "Window @1 not found")
803803
FB_IMPL_MSG(JRD, 802, dsql_window_cant_overr_part, -833, "42", "000", "Cannot use PARTITION BY clause while overriding the window @1")

‎src/jrd/RecordSourceNodes.cpp‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2647,6 +2647,7 @@ void WindowSourceNode::parseWindow(thread_db* tdbb, CompilerScratch* csb)
26472647
{
26482648
case WindowClause::FrameExtent::Unit::RANGE:
26492649
case WindowClause::FrameExtent::Unit::ROWS:
2650+
case WindowClause::FrameExtent::Unit::GROUPS:
26502651
break;
26512652

26522653
default:

‎src/jrd/recsrc/RecordSource.h‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1064,6 +1064,9 @@ namespace Jrd
10641064

10651065
SINT64 locateFrameRange(thread_db* tdbb, Request* request, Impure* impure,
10661066
const Frame* frame, const dsc* offsetDesc, SINT64 position) const;
1067+
SINT64 locateFrameGroups(thread_db* tdbb, Request* request, Impure* impure,
1068+
const Frame* frame, const impure_value_ex* offsetValue, SINT64 position,
1069+
bool startFrame) const;
10671070

10681071
private:
10691072
NestConst<SortNode> m_order;

‎src/jrd/recsrc/WindowedStream.cpp‎

Lines changed: 107 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -225,7 +225,9 @@ WindowedStream::WindowedStream(thread_db* tdbb, Optimizer* opt,
225225
{
226226
// While here, verify not supported functions/clauses.
227227

228-
if (window.order || window.frameExtent->unit == FrameExtent::Unit::ROWS)
228+
if (window.order ||
229+
window.frameExtent->unit == FrameExtent::Unit::ROWS ||
230+
window.frameExtent->unit == FrameExtent::Unit::GROUPS)
229231
{
230232
for (const auto& source : window.map->sourceList)
231233
{
@@ -241,7 +243,9 @@ WindowedStream::WindowedStream(thread_db* tdbb, Optimizer* opt,
241243
if (arg)
242244
{
243245
string msg;
244-
msg.printf("%s is not supported in windows with ORDER BY or frame by ROWS clauses", arg);
246+
msg.printf(
247+
"%s is not supported in windows with ORDER BY or frame by ROWS/GROUPS clauses",
248+
arg);
245249

246250
status_exception::raise(
247251
Arg::Gds(isc_wish_list) <<
@@ -660,7 +664,7 @@ bool WindowedStream::WindowStream::internalGetRecord(thread_db* tdbb) const
660664
if (m_frameExtent->frame1->value && !(m_invariantOffsets & 0x1))
661665
getFrameValue(tdbb, request, m_frameExtent->frame1, &impure->startOffset);
662666

663-
// {range | rows} between unbounded preceding and ...
667+
// {range | rows | groups} between unbounded preceding and ...
664668
// (no order by) range
665669
if ((m_frameExtent->frame1->bound == Frame::Bound::PRECEDING && !m_frameExtent->frame1->value) ||
666670
(!m_order && m_frameExtent->unit == FrameExtent::Unit::RANGE))
@@ -673,12 +677,26 @@ bool WindowedStream::WindowStream::internalGetRecord(thread_db* tdbb) const
673677
{
674678
impure->windowBlock.startPosition = position;
675679
}
680+
// groups between current row and ...
681+
else if (m_frameExtent->unit == FrameExtent::Unit::GROUPS &&
682+
m_frameExtent->frame1->bound == Frame::Bound::CURRENT_ROW)
683+
{
684+
impure->windowBlock.startPosition = locateFrameGroups(tdbb, request, impure,
685+
m_frameExtent->frame1, nullptr, position, true);
686+
}
676687
// rows between <n> {preceding | following} and ...
677688
else if (m_frameExtent->unit == FrameExtent::Unit::ROWS &&
678689
m_frameExtent->frame1->value)
679690
{
680691
impure->windowBlock.startPosition = position + impure->startOffset.vlux_count;
681692
}
693+
// groups between <n> {preceding | following} and ...
694+
else if (m_frameExtent->unit == FrameExtent::Unit::GROUPS &&
695+
m_frameExtent->frame1->value)
696+
{
697+
impure->windowBlock.startPosition = locateFrameGroups(tdbb, request, impure,
698+
m_frameExtent->frame1, &impure->startOffset, position, true);
699+
}
682700
// range between current row and ...
683701
else if (m_frameExtent->unit == FrameExtent::Unit::RANGE &&
684702
m_frameExtent->frame1->bound == Frame::Bound::CURRENT_ROW)
@@ -703,7 +721,7 @@ bool WindowedStream::WindowStream::internalGetRecord(thread_db* tdbb) const
703721
if (m_frameExtent->frame2->value && !(m_invariantOffsets & 0x2))
704722
getFrameValue(tdbb, request, m_frameExtent->frame2, &impure->endOffset);
705723

706-
// {range | rows} between ... and unbounded following
724+
// {range | rows | groups} between ... and unbounded following
707725
// (no order by) range
708726
if ((m_frameExtent->frame2->bound == Frame::Bound::FOLLOWING && !m_frameExtent->frame2->value) ||
709727
(!m_order && m_frameExtent->unit == FrameExtent::Unit::RANGE))
@@ -716,12 +734,26 @@ bool WindowedStream::WindowStream::internalGetRecord(thread_db* tdbb) const
716734
{
717735
impure->windowBlock.endPosition = position;
718736
}
737+
// groups between ... and current row
738+
else if (m_frameExtent->unit == FrameExtent::Unit::GROUPS &&
739+
m_frameExtent->frame2->bound == Frame::Bound::CURRENT_ROW)
740+
{
741+
impure->windowBlock.endPosition = locateFrameGroups(tdbb, request, impure,
742+
m_frameExtent->frame2, nullptr, position, false);
743+
}
719744
// rows between ... and <n> {preceding | following}
720745
else if (m_frameExtent->unit == FrameExtent::Unit::ROWS &&
721746
m_frameExtent->frame2->value)
722747
{
723748
impure->windowBlock.endPosition = position + impure->endOffset.vlux_count;
724749
}
750+
// groups between ... and <n> {preceding | following}
751+
else if (m_frameExtent->unit == FrameExtent::Unit::GROUPS &&
752+
m_frameExtent->frame2->value)
753+
{
754+
impure->windowBlock.endPosition = locateFrameGroups(tdbb, request, impure,
755+
m_frameExtent->frame2, &impure->endOffset, position, false);
756+
}
725757
// range between ... and current row
726758
else if (m_frameExtent->unit == FrameExtent::Unit::RANGE &&
727759
m_frameExtent->frame2->bound == Frame::Bound::CURRENT_ROW)
@@ -765,11 +797,14 @@ bool WindowedStream::WindowStream::internalGetRecord(thread_db* tdbb) const
765797
if (m_exclusion == Exclusion::NO_OTHERS &&
766798
((m_frameExtent->frame1->bound == Frame::Bound::PRECEDING && !m_frameExtent->frame1->value &&
767799
m_frameExtent->frame2->bound == Frame::Bound::FOLLOWING && !m_frameExtent->frame2->value) ||
768-
(m_frameExtent->unit == FrameExtent::Unit::RANGE && !m_order)))
800+
((m_frameExtent->unit == FrameExtent::Unit::RANGE ||
801+
m_frameExtent->unit == FrameExtent::Unit::GROUPS) && !m_order)))
769802
{
770803
impure->rangePending = MAX(0, impure->windowBlock.endPosition - position);
771804
}
772-
else if (m_exclusion == Exclusion::NO_OTHERS && m_frameExtent->unit == FrameExtent::Unit::RANGE)
805+
else if (m_exclusion == Exclusion::NO_OTHERS &&
806+
(m_frameExtent->unit == FrameExtent::Unit::RANGE ||
807+
m_frameExtent->unit == FrameExtent::Unit::GROUPS))
773808
{
774809
SINT64 rangePos = position;
775810
cacheValues(tdbb, request, &m_order->expressions, impure->orderValues,
@@ -1006,7 +1041,8 @@ void WindowedStream::WindowStream::getFrameValue(thread_db* tdbb, Request* reque
10061041
error = true;
10071042
else
10081043
{
1009-
if (m_frameExtent->unit == FrameExtent::Unit::ROWS)
1044+
if (m_frameExtent->unit == FrameExtent::Unit::ROWS ||
1045+
m_frameExtent->unit == FrameExtent::Unit::GROUPS)
10101046
{
10111047
// Purposedly used 32-bit here. So long distance will complicate things for no gain.
10121048
impureValue->vlux_count = MOV_get_long(tdbb, desc, 0);
@@ -1044,6 +1080,14 @@ WindowedStream::WindowStream::Block WindowedStream::WindowStream::getPeerBlock(
10441080
return peerBlock;
10451081
}
10461082

1083+
if ((SINT64) m_next->getPosition(request) != position + 1)
1084+
{
1085+
m_next->locate(tdbb, position);
1086+
1087+
if (!m_next->getRecord(tdbb))
1088+
fb_assert(false);
1089+
}
1090+
10471091
cacheValues(tdbb, request, &m_order->expressions, impure->orderValues, DummyAdjustFunctor());
10481092

10491093
while (peerBlock.startPosition > impure->partitionBlock.startPosition)
@@ -1145,6 +1189,62 @@ bool WindowedStream::WindowStream::isExcluded(SINT64 position, const Block& excl
11451189
position >= exclusion2.startPosition && position <= exclusion2.endPosition);
11461190
}
11471191

1192+
SINT64 WindowedStream::WindowStream::locateFrameGroups(thread_db* tdbb, Request* request,
1193+
Impure* impure, const Frame* frame, const impure_value_ex* offsetValue, SINT64 position,
1194+
bool startFrame) const
1195+
{
1196+
SINT64 offset = 0;
1197+
1198+
if (offsetValue)
1199+
{
1200+
offset = MOV_get_long(tdbb, &offsetValue->vlu_desc, 0);
1201+
1202+
if (frame->bound == Frame::Bound::PRECEDING)
1203+
offset = -offset;
1204+
}
1205+
1206+
Block groupBlock = getPeerBlock(tdbb, request, impure, position);
1207+
1208+
auto restoreAndReturn = [&] (SINT64 result)
1209+
{
1210+
m_next->locate(tdbb, position);
1211+
1212+
if (!m_next->getRecord(tdbb))
1213+
fb_assert(false);
1214+
1215+
return result;
1216+
};
1217+
1218+
if (offset < 0)
1219+
{
1220+
for (SINT64 pending = -offset; pending > 0; --pending)
1221+
{
1222+
if (groupBlock.startPosition <= impure->partitionBlock.startPosition)
1223+
{
1224+
return restoreAndReturn(startFrame ?
1225+
impure->partitionBlock.startPosition : impure->partitionBlock.startPosition - 1);
1226+
}
1227+
1228+
groupBlock = getPeerBlock(tdbb, request, impure, groupBlock.startPosition - 1);
1229+
}
1230+
}
1231+
else if (offset > 0)
1232+
{
1233+
for (SINT64 pending = offset; pending > 0; --pending)
1234+
{
1235+
if (groupBlock.endPosition >= impure->partitionBlock.endPosition)
1236+
{
1237+
return restoreAndReturn(startFrame ?
1238+
impure->partitionBlock.endPosition + 1 : impure->partitionBlock.endPosition);
1239+
}
1240+
1241+
groupBlock = getPeerBlock(tdbb, request, impure, groupBlock.endPosition + 1);
1242+
}
1243+
}
1244+
1245+
return restoreAndReturn(startFrame ? groupBlock.startPosition : groupBlock.endPosition);
1246+
}
1247+
11481248
SINT64 WindowedStream::WindowStream::locateFrameRange(thread_db* tdbb, Request* request, Impure* impure,
11491249
const Frame* frame, const dsc* offsetDesc, SINT64 position) const
11501250
{

0 commit comments

Comments
 (0)