Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions Sulfamerazine/Form_I_thermal_expansion.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
SLFNMA02,,,,,,,
Identifier,Property,Value (1/K),Std, N, Name, Reference,Comment
1, Volumetric thermal expansion coefficient @160K, 0.000119, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
2, Volumetric thermal expansion coefficient @170K, 0.000119, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
3, Volumetric thermal expansion coefficient @180K, 0.00012, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
4, Volumetric thermal expansion coefficient @190K, 0.00012, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
5, Volumetric thermal expansion coefficient @200K, 0.000121, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
6, Volumetric thermal expansion coefficient @210K, 0.000122, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
7, Volumetric thermal expansion coefficient @220K, 0.000122, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
8, Volumetric thermal expansion coefficient @230K, 0.000123, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
9, Volumetric thermal expansion coefficient @240K, 0.000124, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
10, Volumetric thermal expansion coefficient @250K, 0.000124, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
11, Volumetric thermal expansion coefficient @260K, 0.000125, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
12, Volumetric thermal expansion coefficient @270K, 0.000126, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
13, Volumetric thermal expansion coefficient @280K, 0.000126, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
14, Volumetric thermal expansion coefficient @290K, 0.000127, 0.000016, Pallipurath, 10.1021/acs.molpharmaceut.5b00504, SCXRD
15, Volumetric thermal expansion coefficient @310K, 0.000103, 0.000010, , , PXRD
16, Volumetric thermal expansion coefficient @320K, 0.000107, 0.000009, , , PXRD
17, Volumetric thermal expansion coefficient @330K, 0.00011, 0.000008, , , PXRD
18, Volumetric thermal expansion coefficient @340K, 0.000114, 0.000007, , , PXRD
19, Volumetric thermal expansion coefficient @350K, 0.000118, 0.000005, , , PXRD
20, Volumetric thermal expansion coefficient @360K, 0.000122, 0.000005, , , PXRD
21, Volumetric thermal expansion coefficient @370K, 0.000126, 0.000004, , , PXRD
22, Volumetric thermal expansion coefficient @380K, 0.000130, 0.000003, , , PXRD
23, Volumetric thermal expansion coefficient @390K, 0.000134, 0.000003, , , PXRD
24, Volumetric thermal expansion coefficient @400K, 0.000137, 0.000004, , , PXRD
25, Volumetric thermal expansion coefficient @410K, 0.000141, 0.000004, , , PXRD
26, Volumetric thermal expansion coefficient @420K, 0.000145, 0.000005, , , PXRD
27, Volumetric thermal expansion coefficient @430K, 0.000149, 0.000006, , , PXRD
28, Volumetric thermal expansion coefficient @440K, 0.000152, 0.000007, , , PXRD
29, Volumetric thermal expansion coefficient @450K, 0.000156, 0.000008, , , PXRD
30, Volumetric thermal expansion coefficient @460K, 0.000160, 0.000009, , , PXRD
31, Volumetric thermal expansion coefficient @470K, 0.000164, 0.000011, , , PXRD
120 changes: 120 additions & 0 deletions Sulfamerazine/Supplemetary_data/thermal_expansion_coeffecient.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# import required libraries

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant comment I guess

import pandas as pd
import numpy as np
import sys
from scipy.stats import t

# User parameters
INPUT_CSV = sys.argv[1] # Change as needed

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are their going to be instructions on how to run this on the command line? I guess the user won't change this line, rather they are free to specify different filenames which are picked up by the script.

TEMP_COL = 'Temperature/ K'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm generally not a fan of units in column headers. These can be defined elsewhere, even in a separate 'code book'.

VOLUME_COL = 'volume / Angstrom-3'
NOTES_COL = 'notes'
INCREMENT = 10 # Kelvin
if "Form" in INPUT_CSV:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Form could be a string constant FORM = 'Form', and re-used.

name = f"Form_{INPUT_CSV.split('_')[1]}_"
else:
name = ""
# Output file names
OUTPUT_SCXRD = f'thermal_expansion_coefficient_{name}SCXRD.csv'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thermal_expansion_coefficient could be a string constant

OUTPUT_PXRD = f'thermal_expansion_coefficient_{name}PXRD.csv'
OUTPUT_ALL = f'thermal_expansion_coefficient_{name}ALL.csv'
# Read the data
try:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check could be moved earlier, saving the previous few lines of execution which might then be redundant if exception thrown here.

df = pd.read_csv(INPUT_CSV)
except Exception as e:
print(f"Error reading '{INPUT_CSV}': {e}")
exit(1)

df.columns = df.columns.str.strip()

required_cols = [TEMP_COL, VOLUME_COL, NOTES_COL]
missing_cols = [col for col in required_cols if col not in df.columns]
if missing_cols:
print(f"Error: The following required columns are missing from '{INPUT_CSV}': {missing_cols}")
print("Please ensure your CSV contains columns named exactly as follows:")
for col in required_cols:
print(f" - {col}")
print("Or, modify this script to match your column names.")
exit(1)
# Helper function to process a subset
def process_and_save(sub_df, output_csv):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <302> reported by reviewdog 🐶
expected 2 blank lines, found 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <302> reported by reviewdog 🐶
expected 2 blank lines, found 0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The functions could have docstrings

# Drop rows with missing temperature or volume
sub_df = sub_df.dropna(subset=[TEMP_COL, VOLUME_COL])
if len(sub_df) < 3:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could give a hint that missing data was found which might prevent some head scratching if the script exits for this reason.

print(f"Not enough data for {output_csv}")
return
temps = sub_df[TEMP_COL].astype(float).values
vols = sub_df[VOLUME_COL].astype(float).values
coeffs = np.polyfit(temps, vols, 2)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think np.Polynomial is now preferred to polyfit
https://numpy.org/doc/stable/reference/routines.polynomials.html

p = np.poly1d(coeffs)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variables with single letter names can be a pain to search for and p isn't very disambiguating (maybe there's a p-value later, or a pressure etc.)


def expansion_coefficient(T):
return 2*coeffs[0]*T + coeffs[1]

T_min = int(np.ceil(temps.min() / INCREMENT) * INCREMENT)
T_max = int(np.floor(temps.max() / INCREMENT) * INCREMENT)
T_points = np.arange(T_min, T_max + 1, INCREMENT)
exp_coeffs = expansion_coefficient(T_points)
volumes_at_points = np.polyval(coeffs, T_points)
norm_exp_coeffs = exp_coeffs / volumes_at_points

# Confidence band calculation

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confidence interval rather than band. And say what it's a CI for, is it for the expansion coefficients?

# Fit quadratic: vols = a*T^2 + b*T + c

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fitting already occurred.

# Get predicted values
y_pred = p(temps)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's predicting volumes, I might make the variable name appropriate to the context (vols_pred?)

residuals = vols - y_pred
dof = len(temps) - 3 # 3 parameters for quadratic
s_err = np.sqrt(np.sum(residuals**2) / dof)

# For each T_point, calculate confidence interval
# Design matrix for quadratic
X = np.vstack([T_points**2, T_points, np.ones_like(T_points)]).T

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[flake8] <841> reported by reviewdog 🐶
local variable 'X' is assigned to but never used

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like flake said, X is not used.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[flake8] <841> reported by reviewdog 🐶
local variable 'X' is assigned to but never used

X_data = np.vstack([temps**2, temps, np.ones_like(temps)]).T

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On this section of code, quite a few comments.
[1] I think it's trying to compute a 95% confidence interval for a predicted mean because in line 82 there is no +1 inside the sqrt. It looks kind of OK but it constructs things a bit back to front for me (temps**2 as the first column etc in X_data. It'll be fine if everything is consistently ordered).
[2] conf_band looks like a list of numbers rather than intervals. I'd expect the interval to be [y-conf_band, y+conf_band] but I guess the output dataframe only claims a (+/-) here so it's up to the user to do the arithmetic. The script could be changed to do this for them as in actually output [y-conf_band, y+conf_band] .
[3] Unclear what the norm_conf_band is doing but maybe output 'y' is divided by volumes, so the conf interval on y would need to be adjusted to generate a conf. interval on y/volume.
[4] As said, it looks sort of OK but it's doing it all 'by hand' and it would be good to verify against data for which we know the answer. On this point, there is a python model statsmodels. We could try to use this? It seems fairly straighforward to get both confidence and prediction intervals, and fit diagnostics.
Here is some python do fit a quadratic using statsmodels and get confidence and prediction intervals. I am confident about this as I managed to reproduce the exact same confidence and prediction intervals in R.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
 import statsmodels.formula.api as smf
# Generate synthetic data
np.random.seed(0)
n = 10
x = np.linspace(0, 10, n)
e = np.random.normal(size=n)

y = 1 + 0.5 * x + 0.8*x**2 + 2 * e
print(f'y {y}')
X = pd.DataFrame({'x':x})
print(f'X {X}')

model = smf.ols(formula = 'y ~ x + np.power(x, 2)', data = {'y':y, 'x':X}).fit()
print(model.summary())
pred = model.get_prediction(X)
pred_summary = pred.summary_frame(alpha=0.05)  # 95% confidence intervals

# Extract confidence intervals
ci_lower = pred_summary['mean_ci_lower']
ci_upper = pred_summary['mean_ci_upper']
pi_lower = pred_summary['obs_ci_lower']
pi_upper = pred_summary['obs_ci_upper']

plt.scatter(x, y, label='Data')

plt.plot(x, model.fittedvalues, color='red', label='Fitted Line')
plt.fill_between(x, ci_lower, ci_upper, color='red', alpha=0.3, label='95% CI')
plt.fill_between(x, pi_lower, pi_upper, color='blue', alpha=0.2, label='95% PI')

plt.xlabel('X')
plt.ylabel('Y')
plt.legend()
plt.show()

image

# (X^T X)^-1
XT_X_inv = np.linalg.inv(X_data.T @ X_data)
# t-value for 95% confidence
t_val = t.ppf(0.975, dof)

conf_band = []
for i, T in enumerate(T_points):
x0 = np.array([T**2, T, 1])
# Standard error of prediction
se_pred = s_err * np.sqrt(x0 @ XT_X_inv @ x0.T)
conf_band.append(t_val * se_pred)
conf_band = np.array(conf_band)
norm_conf_band = conf_band / volumes_at_points
output_df = pd.DataFrame({
'Identifier': np.arange(1, len(T_points) + 1),
'Property': [f"Volumetric thermal expansion coefficient @{int(T)}K" for T in T_points],
# 'Thermal Expansion Coefficient (dV/dT)': exp_coeffs,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <122> reported by reviewdog 🐶
continuation line missing indentation or outdented

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <122> reported by reviewdog 🐶
continuation line missing indentation or outdented

'Value (1/K)': norm_exp_coeffs,
# 'Regression Fit (Volume)': volumes_at_points,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <122> reported by reviewdog 🐶
continuation line missing indentation or outdented

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <122> reported by reviewdog 🐶
continuation line missing indentation or outdented

# 'Confidence Band (+/-)': conf_band,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <122> reported by reviewdog 🐶
continuation line missing indentation or outdented

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <122> reported by reviewdog 🐶
continuation line missing indentation or outdented

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this means the user has to + and - this value against the appropriate predicted value.

'Std': norm_conf_band,
'N': "",
'Name': "",
'Reference': "",
'Comment': ""
})
output_df.to_csv(output_csv, index=False)

# Separate SC-XRD and PXRD data
df[NOTES_COL] = df[NOTES_COL].astype(str)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <305> reported by reviewdog 🐶
expected 2 blank lines after class or function definition, found 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [flake8] <305> reported by reviewdog 🐶
expected 2 blank lines after class or function definition, found 1

scxrd_df = df[df[NOTES_COL].str.contains('SC-XRD', case=False, na=False)]
pxrd_df = df[df[NOTES_COL].str.contains('PXRD', case=False, na=False)]

has_scxrd = not scxrd_df.empty
has_pxrd = not pxrd_df.empty

if has_scxrd or has_pxrd:
if has_scxrd:
process_and_save(scxrd_df, OUTPUT_SCXRD)
else:
print("No SC-XRD data found. Explicitly labelled 'SC-XRD' in the Notes column")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this print statement text express what you want?

if has_pxrd:
process_and_save(pxrd_df, OUTPUT_PXRD)
else:
print("No PXRD data found. Explicitly labelled 'PXRD' in the Notes column")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this print statement text express what you want?

else:
print("No Explicitly labelled SC-XRD or PXRD data found. Processing all data together.")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

capital E on explicitly

process_and_save(df, OUTPUT_ALL)
Loading