Add element_type/name_pattern/level filters to get_elements, get_element_names, get_elements_dataframe#1401
Open
meyersrl wants to merge 3 commits into
Conversation
…element_types) Adds two private module-level helpers in ElementService.py that translate optional element_type / name_pattern / level filters into OData $filter clauses. No public API change yet — subsequent commits wire these into get_elements, get_element_names, and get_elements_dataframe. Includes ~50 pure-function unit tests covering translation, type coercion, glob-to-OData conversion (startswith/endswith/contains/multi-segment), OData single-quote escaping, and all validation error paths.
…d get_element_names
Adds three optional, AND-composable filter kwargs to the two list-returning
element retrieval methods. Filters translate to OData $filter server-side
via _build_elements_filter (introduced in the previous commit).
element_type accepts Element.Types enum, str ('numeric'/'string'/
'consolidated', case-insensitive), int (1/2/3), or an iterable of those
(OR-combined). name_pattern is a glob with '*' wildcard, case- and
space-insensitive (matching the semantics of get_elements_filtered_by_wildcard).
level is an exact match on the hierarchy level integer.
All three default to None, purely additive, no breaking changes. Includes
integration tests against a fixture dimension covering each kwarg alone,
in combination, with case/space variations, quote-escape, and validation
error paths.
… trio kwargs to get_elements_dataframe get_elements_by_level and get_elements_filtered_by_wildcard are now thin delegations to get_element_names with the appropriate kwargs. Behavior is preserved (verified by regression tests against snapshots captured from master before the refactor). Net effect: single source of truth for OData $filter construction across the four element-listing methods. get_elements_dataframe gains element_type / name_pattern / level kwargs. When any of the trio is set while elements is None, the method resolves the selection via get_element_names and feeds it into the existing MDX path. The trio is authoritative and overrides skip_consolidations (documented in the docstring).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three optional, AND-composable filter kwargs —
element_type,name_pattern,level— toget_elements,get_element_names, andget_elements_dataframe. Translates to OData$filterserver-side. Refactorsget_elements_by_levelandget_elements_filtered_by_wildcardto delegate to the new path (behavior preserved).Motivation
Today TM1py can count elements by type (
get_number_of_leaf_elements,get_number_of_consolidated_elements, etc.) but can't list them without pulling the entire dimension and filtering in Python, or hand-writing MDX. This closes the gap with a small, additive API.API
element_typeacceptsElement.Typesenum, str ('numeric'/'string'/'consolidated', case-insensitive), int (1/2/3), or an iterable of any of those (OR-combined).name_patternis a glob with*wildcard, case- and space-insensitive (matching the semantics of the existingget_elements_filtered_by_wildcard). Translates tostartswith,endswith,contains, oreqdepending on*placement.?raisesValueError.levelis exact match on the hierarchy level (0 = leaf).All three default to
None— purely additive, no breaking changes.Design notes worth flagging
1. Incidental fix in get_elements URL. The original
get_elementsURL used?select=Name,Type(missing the$prefix), which TM1's OData endpoint silently ignored — the response always carried the full element payload. This PR corrects it to?$select=Name,Type.Element.from_dicthandlesUniqueName,Index, andAttributesvia.get(..., None)so no existing caller breaks. Same latent bug exists atget_edges(line 706) but is out of scope for this PR.2. element_type overrides skip_consolidations. In
get_elements_dataframe, whenelement_typeis explicitly set alongsideskip_consolidations=True,element_typeis authoritative. The two kwargs are not AND-combined — this is a deliberate design choice documented in the docstring. (Settingelement_type=['numeric','consolidated']while leavingskip_consolidations=Truewould otherwise be incoherent: you asked for consolidations but the default flag drops them.)3. Empty-match dataframe schema. When the trio filter matches zero elements,
get_elements_dataframeconstructs an empty MDX set viaTm1FilterByLevel({Members}, 9999)rather than the bare{}. The bare empty set loses the dimension column in the downstreampd.merge, breaking the DataFrame schema for callers that index by attribute or level column. The 9999-level filter produces an empty result while preserving the full column schema. A regression test (test_dataframe_trio_empty_match_preserves_schema) guards this.Test plan
_coerce_element_types,_build_elements_filter) inTests/ElementService_filtering_helpers_test.py, covering translation, type coercion, glob-to-OData conversion, single-quote escaping, and every validation error path. No TM1 connection required.Tests/ElementService_test.py::TestElementFilteringagainst a fixture dimension covering each kwarg alone, in combination, with case/space variations, quote-escaped names (O'Brien), and validation passthrough.get_elements_by_levelandget_elements_filtered_by_wildcardproduce identical output tomaster(snapshots committed underTests/fixtures/element_filtering_snapshots/).get_elements_dataframeproduces an identical DataFrame tomasterwhen no trio kwargs are passed.Total ~85 new tests. Verified against a live TM1 11.8.x server: all green, no regressions in the existing 115
TestElementServicetests.Backwards compatibility
Purely additive. All new kwargs default to
None. No signatures removed, no return types changed, no error semantics changed for any existing path.Files touched
TM1py/Services/ElementService.py— two new private helpers, kwargs added to three public methods, two existing methods refactored to thin delegations.Tests/ElementService_filtering_helpers_test.py— new file, unit tests for the helpers.Tests/ElementService_test.py— newTestElementFilteringclass.Tests/fixtures/element_filtering_snapshots/— regression oracle snapshots (9 files).