Backwards incompatible change in Filter in 6.40

[acampove]

Check duplicate issues.

• Checked for duplicates

Description

I have always used (1) as a placeholder for skipping a cut and that is all over the code I use. It seem that both (1) and 1 are not valid anymore?

(rx_run3) [acampove@thinkbook ~]$ ipython
Python 3.13.0 | packaged by conda-forge | (main, Nov 27 2024, 19:18:50) [GCC 13.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 9.13.0 -- An enhanced Interactive Python. Type '?' for help.
Tip: `?` alone on a line will brings up IPython's help

In [1]: from ROOT import RDataFrame
[TerminalIPythonApp] WARNING | Shell is already running a different gui event loop for ROOT. Call with no arguments to disable the current loop.
[TerminalIPythonApp] WARNING | Shell is already running a different gui event loop for ROOT. Call with no arguments to disable the current loop.
[TerminalIPythonApp] WARNING | Shell is already running a different gui event loop for ROOT. Call with no arguments to disable the current loop.
[TerminalIPythonApp] WARNING | Shell is already running a different gui event loop for ROOT. Call with no arguments to disable the current loop.
[TerminalIPythonApp] WARNING | Shell is already running a different gui event loop for ROOT. Call with no arguments to disable the current loop.

In [2]: rdf = RDataFrame(100)

In [3]: rdf = rdf.Define('a', 'gRandom->Gaus()')

In [4]: rdf = rdf.Filter('(1)', 'x')
---------------------------------------------------------------------------
runtime_error                             Traceback (most recent call last)
Cell In[4], line 1
----> 1 rdf = rdf.Filter('(1)', 'x')

File ~/micromamba/envs/rx_run3/lib/python3.13/site-packages/ROOT/_pythonization/_rdf_pyz.py:404, in _PyFilter(rdf, callable_or_str, extra_args, *args)
    368 """
    369 Filters the entries of RDF according to a given condition.
    370 Arguments:
   (...)    401 
    402 """
    403 if isinstance(callable_or_str, str):  # If string argument is passed. Invoke the Original Filters.
--> 404     return rdf._OriginalFilter(callable_or_str, *args)
    405 # The 1st argument is either a string or a python callable.
    406 if not callable(callable_or_str):

runtime_error: Template method resolution failed:
  ROOT::RDF::RInterface<ROOT::Detail::RDF::RJittedFilter> ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager>::Filter(string_view expression, string_view name = "") =>
    runtime_error: Filter: the following expression does not evaluate to bool:
(1)
  ROOT::RDF::RInterface<ROOT::Detail::RDF::RJittedFilter> ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager>::Filter(string_view expression, string_view name = "") =>
    runtime_error: Filter: the following expression does not evaluate to bool:
(1)

In [5]: rdf = rdf.Filter('1', 'x')
---------------------------------------------------------------------------
runtime_error                             Traceback (most recent call last)
Cell In[5], line 1
----> 1 rdf = rdf.Filter('1', 'x')

File ~/micromamba/envs/rx_run3/lib/python3.13/site-packages/ROOT/_pythonization/_rdf_pyz.py:404, in _PyFilter(rdf, callable_or_str, extra_args, *args)
    368 """
    369 Filters the entries of RDF according to a given condition.
    370 Arguments:
   (...)    401 
    402 """
    403 if isinstance(callable_or_str, str):  # If string argument is passed. Invoke the Original Filters.
--> 404     return rdf._OriginalFilter(callable_or_str, *args)
    405 # The 1st argument is either a string or a python callable.
    406 if not callable(callable_or_str):

runtime_error: Template method resolution failed:
  ROOT::RDF::RInterface<ROOT::Detail::RDF::RJittedFilter> ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager>::Filter(string_view expression, string_view name = "") =>
    runtime_error: Filter: the following expression does not evaluate to bool:
1
  ROOT::RDF::RInterface<ROOT::Detail::RDF::RJittedFilter> ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager>::Filter(string_view expression, string_view name = "") =>
    runtime_error: Filter: the following expression does not evaluate to bool:
1

In [6]: rdf = rdf.Filter('true', 'x')

In [7]: 

• was this intentional or a mistake?
• How should we add 'no cut' cuts?
• Will 'true' be supported in the long term?

Reproducer

from ROOT import RDataFrame

rdf = RDataFrame(100)
rdf = rdf.Define('a', 'gRandom->Gaus()')
rdf = rdf.Filter('(1)', 'x')

ROOT version

(rx_run3) [acampove@thinkbook ~]$ root --version
ROOT Version: 6.40.02
Built for linuxx8664gcc on Jun 14 2026, 10:42:24
From tags/6-40-02@6-40-02
(rx_run3) [acampove@thinkbook ~]$

Installation method

micromamba

Operating system

almalinux 8

Additional context

No response

[vepadulano]

Dear @acampove ,

Thank you for your report!

In this case your code has always been wrong: Filter expects an expression that returns naturally the type bool. When you pass the string "1" or "(1)", internally RDF creates an equivalent C++ functions that would look like

auto foo() { return 1; }
auto bar() { return (1); }

In standard C++ neither of these functions returns naturally bool, as you can see e.g. at https://godbolt.org/z/s15s7oKe5 .

The reason why it seemed to "work" in the past is that the error generated here

root/tree/dataframe/src/RDFInterfaceUtils.cxx

Line 630 in 33dbe67

std::runtime_error("Filter: the following expression does not evaluate to bool:\n" + std::string(expression));

was not properly reported.

The good news is that thanks to 3e15cad the error is now being properly thrown. So, RDataFrame is even safer than before and lets you know that your code was wrong!

Even better, you can fix your code by simply using standard C++ behaviour: if your expression is "true", that will be converted internally to the equivalent function:

auto baz() { return true; }

Which does naturally return bool.

[acampove]

@vepadulano

I do not see it that way. The way I see it, ROOT had a backwards incompatible change that broke the users code, If my code had been "wrong" I would have got the wrong results and I would have figured out the wrongness myself.

Regarding:

auto baz() { return true; }

for your education, most of us do not use c++ for physics analysis. A more useful snippet would have been

rdf = rdf.Filter('true', 'some label')

Can you confirm that you will not break that too and we can use it in the long term?

Cheers.

[vepadulano]

Dera @acampove ,

Thanks for your reply!

ROOT had a backwards incompatible change that broke the users code

I acknowledge you have an opinion. Nothing was broken here, ROOT was made safer. I hope you generally prefer the software libraries you use to make your code safer and less bug-prone.

for your education

Thanks for educating me! I confirm, as I wrote explicitly in my previous message, that true is the standard C++ keyword for the boolean true value, so you can write 'true' as a Filter expression. There is no aim in the long term to go against the standard implementation of any of the programming languages supported by the ROOT project (including Python and C++). I understand the true keyword is quite different from the True keyword used by the C implementation of the Python programming language, so I hope that this is now clearer.