Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
d25abaf
Added jpk-qi-data loading functionality. Two possible channels: by tr…
ahobbs7 Feb 12, 2026
9d28dec
Adding get available channels function to each file format as well as…
ahobbs7 Feb 16, 2026
4bd6bb3
Making load jpk qi data function use zipfile instead of afmformats
ahobbs7 Feb 23, 2026
28f7633
Fixing scaling
ahobbs7 Feb 23, 2026
a67b11f
Updating general_loader and dependencies
ahobbs7 Feb 23, 2026
37e1ae2
Adjusting h5_jpk so it works for different shaped images
ahobbs7 Feb 23, 2026
16a827c
Making the jpk-qi-data processing save the h5 jpk file in the correct…
ahobbs7 Feb 23, 2026
c780de5
Add the ability to save the metadata to h5 from jpk qi. Additionally …
ahobbs7 Feb 25, 2026
a764ca6
Adding ability to load force curves from h5 file
ahobbs7 Feb 27, 2026
405a95d
Improving speed of loading qi curve data by loading all the curves at…
ahobbs7 Feb 27, 2026
3198567
Making loading jpk-qi-data return all the curves
ahobbs7 Feb 27, 2026
c25fb75
Adding .bin files support
ahobbs7 Mar 9, 2026
085b252
Adding returning of metadata
ahobbs7 Mar 9, 2026
06ec0de
Refactoring jpk-qi-data reader to use a loader class for greater modu…
ahobbs7 Mar 12, 2026
78c54a7
Starting to implement lazy loading
ahobbs7 Mar 16, 2026
3c327e9
Made force curves lazy loaded for h5-jpk and jpk-qi-data
ahobbs7 Mar 16, 2026
e776c66
Implementing caching of heavy data objects/ references to large open …
ahobbs7 Mar 18, 2026
3e0b8b1
Adjusting curve data access method to work more like a 2D array for m…
ahobbs7 Mar 23, 2026
b8ec43a
Separating saving functionality for jpk-qi-data and adding function t…
ahobbs7 Mar 25, 2026
10c1050
Fixing duplicated converting to nm bug
ahobbs7 Mar 26, 2026
b510141
Minor changes to fix double scaling on current channel as well
ahobbs7 Mar 26, 2026
afc65cc
Fixing minor error
ahobbs7 Mar 27, 2026
b1ec527
Added timing for testing and started converting to more memory and ti…
ahobbs7 Mar 30, 2026
1fd1eed
Making jpk-qi-data loading stream data into h5 file rather than savin…
ahobbs7 Mar 30, 2026
4e90188
Changing metdata data saving so 'changing keys' are assumed based on …
ahobbs7 Apr 1, 2026
66f388a
Pre-sizing the curve data to make loading faster (using a best guess)
ahobbs7 Apr 1, 2026
f286e57
Improving performance by removing javaproperties reliance in loop
ahobbs7 Apr 1, 2026
5391179
Making the saving to h5 save in sections using a buffer
ahobbs7 Apr 2, 2026
f0235ad
Removing redundant functions and fixing minor index bugs
ahobbs7 Apr 2, 2026
62481da
removing possibility of size 1 image stack
ahobbs7 Apr 3, 2026
588c725
adding comments and formatting
ahobbs7 Apr 3, 2026
4c7ea88
Ensuring that the h5-jpk copy of the jpk-qi-data doesn't get overwritten
ahobbs7 Apr 24, 2026
62637c2
Improving __iter__ function so loading of all curves for analysis is …
ahobbs7 Apr 24, 2026
b9104cc
Fixing errors with tests caused by logging problems
ahobbs7 Apr 24, 2026
0a1f459
Updating tests for jpk-qi-data and h5-jpk with curve data
ahobbs7 Apr 24, 2026
58a3a0f
Updating documentation
ahobbs7 Apr 24, 2026
c0a2924
Reformatting to match pre-commit conditions and make more robust
ahobbs7 May 2, 2026
e646205
Minor formatting changes on tests
ahobbs7 May 2, 2026
a6388fd
Indices spelling correction
ahobbs7 May 2, 2026
b88ee26
Skipping tests requiring large test files which cannot be added to repo
ahobbs7 May 3, 2026
be2840e
[pre-commit.ci] Fixing issues with pre-commit
pre-commit-ci[bot] May 3, 2026
a1764fe
Fixing pre-commit problems
ahobbs7 May 4, 2026
603d7bf
chore: removing print statements
ahobbs7 May 22, 2026
43640ac
tests: adding tests for get channel functions
ahobbs7 May 23, 2026
0146542
fix: stopped unnecessary redefining of nested row proxy classes
ahobbs7 May 23, 2026
0f6dabf
docs: comments and improving variable naming
ahobbs7 May 28, 2026
612d4e9
refactor: simplifying dataclass structure by making them work closer …
ahobbs7 Jun 1, 2026
b104b3b
tests: updating curve loading tests (currently being skipped) for ref…
ahobbs7 Jun 1, 2026
2394556
chore: renaming get_pixel_metadata to get_point_metadata
ahobbs7 Jun 1, 2026
699c55b
feat: moving channel_units to be stored per volume rather than in met…
ahobbs7 Jun 1, 2026
773daca
refactor: using a dataclass to represent all return data consistently
ahobbs7 Jun 5, 2026
0b29bbb
refactor: moving jpk-qi-data logic in general_loader into jpk_qi
ahobbs7 Jun 5, 2026
23523fb
chore: moving channels logic out of general_loader and into topostats
ahobbs7 Jun 5, 2026
724460f
fix: resizing of datasets while saving not including current buffered…
ahobbs7 Jun 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ __pycache__/
*.py[cod]
*$py.class

AFMReader/data/*
AFMReader/notebooks/*

# C extensions
*.so

Expand Down
58 changes: 45 additions & 13 deletions AFMReader/asd.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import numpy.typing as npt
from matplotlib import animation

from AFMReader.data_classes import AFMLoad
from AFMReader.io import (
read_ascii,
read_bool,
Expand Down Expand Up @@ -182,7 +183,7 @@ def calculate_scaling_factor(
raise ValueError(f"channel {channel} not known for .asd file type.")


def load_asd(file_path: str | Path, channel: str):
def load_asd(file_path: str | Path, channel: str) -> AFMLoad:
"""
Load a .asd file.

Expand All @@ -196,17 +197,16 @@ def load_asd(file_path: str | Path, channel: str):

Returns
-------
npt.NDArray
The .asd file frames data as a numpy 3D array N x W x H
(Number of frames x Width of each frame x height of each frame).
float
The number of nanometres per pixel for the .asd file. (AKA the resolution).
Enables converting between pixels and nanometres when working with the data, in order to use real-world length
scales.
dict
Metadata for the .asd file. The number of entries is too long to list here, and changes based on the file
version please either look into the `read_header_file_version_x` functions or print the keys too see what
metadata is available.
AFMLoad
An AFMLoad object containing:
- image : npt.NDArray
Shape (Number of frames x Width of each frame x height of each frame).
- px2nm : float
The number of nanometres per pixel for the .asd file.
- metadata : dict
Metadata for the .asd file. The number of entries is too long to list here, and changes based on the file
version please either look into the `read_header_file_version_x` functions or print the keys too see what
metadata is available.
"""
# Ensure the file path is a Path object
file_path = Path(file_path)
Expand Down Expand Up @@ -285,7 +285,39 @@ def load_asd(file_path: str | Path, channel: str):
frames = np.array(frames)

logger.info(f"[{filename}] : Extracted image.")
return frames, pixel_to_nanometre_scaling_factor, header_dict
return AFMLoad(image=frames, px2nm=pixel_to_nanometre_scaling_factor, metadata=header_dict)


def get_asd_channels(file_path: Path):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have time to check this manually - does it work? There isn't a test but frankly we don't have the dev time to move slowly. If you say it works, this is fine with me.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The channel fetching seems to work though I'm happy to quickly make some tests for them as that should be pretty quick.

"""
Get the channels available in given .asd file.

Parameters
----------
file_path : Path
Path to the .asd file.

Returns
-------
list
List of channels available in the .asd file.
"""
with Path.open(file_path, "rb", encoding=None) as open_file: # pylint: disable=unspecified-encoding
file_version = read_file_version(open_file)

if file_version == 0:
header_dict = read_header_file_version_0(open_file)

elif file_version == 1:
header_dict = read_header_file_version_1(open_file)

elif file_version == 2:
header_dict = read_header_file_version_2(open_file)
else:
raise ValueError(
f"File version {file_version} unknown. Please add support if you know how to decode this file version."
)
return [header_dict["channel1"], header_dict["channel2"]]


def read_file_version(open_file: BinaryIO) -> int:
Expand Down
Loading
Loading