Modelling-Preferences-on-Street-Redesign:

A Perception-Driven Framework Using Visual and Segmentation Data

------------------------------------------------------------------

Setup and Requirements

This repository mixes Python (image processing / segmentation) and R (survey + SEM analysis).

Python environment

• Python > 3.8
• PyTorch (with CUDA/cuDNN if GPU training is used)
• torchvision
• opencv-python
• numpy
• pandas
• scikit-learn
• matplotlib

Create a virtual environment and install:
pip install torch torchvision opencv-python numpy pandas scikit-learn matplotlib

R environment

• R > 4.2
• psych (EFA)
• GPArotation (rotations for factor analysis)
• lavaan (CFA and SEM)
• semTools (bootstraps, reliability)
• polycor (polychoric correlations)
• dplyr, tidyr, purrr (data wrangling)
• ggplot2 (plots)

Install directly in R:
install.packages(c("psych", "GPArotation", "lavaan", "semTools", "polycor", "dplyr", "tidyr", "purrr", "ggplot2"))

Note: These are the main requirements used across the scripts. Some scripts may import additional utilities (e.g. plotting or file I/O helpers). Check each script header for a complete list of imports.

Graphical Abstract

[Graphical Abstract.pdf](Graphical Abstract.pdf) is designed as a quick-reference guide, complementing the Workflow Summary bellow. For easier reference, the abstract is also split into numbered main sections so readers can either view the whole pipeline at once or focus on specific steps in isolation.

Workflow Summary

0_Image Pair Collection

To avoid copyright restrictions from Google Street View, all before/after street image pairs were collected from open or self-produced sources:

• Mapillary & KartaView: crowdsourced street-level photos;
• Official project websites/articles (e.g., ArchDaily, Planetizen, municipal news): documented street transformation projects;
• Own photos: taken when other material was unavailable;
• Manual edits: in cases where only a before image was available, I edited the image to approximate the after condition (and vice-versa)
• Filtering: pairs with no public source images found were removed

Each pair represents a real-world intervention, forming the dataset for subsequent color and segmentation analysis.

0_Colour Data Gathering:

[ColourAnalysis.py](0_Colour Data/ColourAnalysis.py) processes paired before-after street images to extract measures of visual change:

1. Image Preprocessing

• Pairs up corresponding before/after images.
• Resizes and aligns them for fair comparison.

2. General Colour Change

• Converts images into CIE LAB colour space.
• Computes pixel-level colour difference (ΔE2000).
• Outputs summary statistics (mean, std, % of pixels with large change).
• Saves ΔE2000 heatmaps and binary masks showing where change occurred.

3. Colour Presence Analysis

• Extracts proportions of 6 colour categories (red, orange, yellow, green, blue, purple).
• Computes diversity of colours (entropy measure).
• Saves per-image masks (where each colour appears) and bar plots comparing before/after.

4. Brightness Analysis

• Computes mean, standard deviation, and IQR of the brightness channel (V in HSV).
• Exports pixel-level brightness masks and histograms with summary stats.

5. Saturation Analysis

• Computes mean and std of saturation channel (S in HSV).
• Saves pixel-level saturation masks and histograms.

6. Outputs

• CSV files with all per-image metrics (general_colour_analysis.csv, colour_presence_analysis.csv, brightness_analysis.csv, saturation_analysis.csv).
• Diagnostic plots (heatmaps, masks, histograms, bar plots).

This dataset forms the colour-derived features later linked with survey perceptions and SEM modeling.

0_Segmentation Model Training

[CityscapesCNN - Train.py](0_Segmentation Data/CityscapesCNN - Train.py) fine-tunes a DeepLabV3 model with ResNet-101 backbone on the Cityscapes dataset, producing the trained model weights used later for inference and comparison.

Key Components

• Model: torchvision.models.segmentation.deeplabv3_resnet101
  • Pretrained on COCO, final classifier replaced with 19 relevant classes
• Dataset: Cityscapes gtfine and leftImg8bit, with label remapping (34 to 19 classes)
  • Custom synchronized data augmentation (crop, horizontal flip and color jitter)
• Loss function: Weighted cross-entropy (class balancing)
• Training setup:
  • Mixed-precision (torch.amp)
  • AdamW optimizer + OneCycleLR scheduler
  • Gradient accumulation for stability
  • Early stopping based on validation IoU

Outputs

• Saved model weights (.pth) for later inference and comparison scripts
• Training logs (loss curves, IoU evolution)
• Best validation checkpoint automatically stored

The result of this step is a ready-to-use segmentation model fine-tuned for analyzing urban street scenes.

0_Segmentation Inference and Comparison

CityscapesCNN - Inference.py and CityscapesCNN - Predictions Comparison.py handle the application of the trained DeepLabV3 model to the selected before/after street images, and the subsequent quantitative analysis of changes in urban elements.

Inference

CityscapesCNN - Inference.py

• Loads the trained model weights (.pth) from the training step
• Runs inference on all images in the dataset (B4nAfter - Chosen)
• Produces colored segmentation masks with the Cityscapes color palette
• Resizes predictions back to the original image resolution
• Saves results to B4nAfter_Predictions_Uncorrected
• Also exports a legend figure (Cityscapes_Legend.png) showing all 19 classes

Comparison

CityscapesCNN - Predictions Comparison.py

• Reads the corrected prediction masks from B4nAfter_Predictions_Corrected
• Groups images into before/after pairs based on file naming convention
• Counts pixel distribution by class (road, sidewalk, building, vegetation, sky, person, vehicle, bicycle, cycle lane, bus lane)
• Computes Δ(after-before) class proportions for each pair
• Generates a bar plot per pair showing pixel % changes
• Exports a full CSV dataset (before_after_pixel_change.csv) with all before/after proportions and deltas

Together, these scripts provide both visual evidence (segmentation masks, plots) and quantitative measures (CSV tables) of how urban elements shifted between before and after street transformations.

1_Univariate Analysis

All survey, segmentation, and colour data were merged into a single Excel file ([0 - All Data and Analysis.xlsx](0 - All Data and Analysis.xlsx)) for descriptive exploration. This step served as a foundation before moving into bivariate and multivariate modelling.

Continuous Variables

For each continuous feature (e.g., age, commute_time, road_spc_var, deltaE_mean), the following statistics were computed:

• Central tendency: mean, median
• Spread: standard deviation, min, max, range, interquartile range (IQR)
• Shape: skewness
• Data quality: number of outliers

This allowed a first look at the variability and potential data cleaning needs across demographic, perceptual, and image-derived variables.

Likert-Scale Perception Variables

Perception items measured on 5-point Likert scales (comfort, safety, liveability, order, stress, security, usefulness) were summarised with:

• Central tendency: mode, median
• Spread: minimum, maximum, range, interquartile range (Q1, Q3)

This approach respected the ordinal nature of the variables while still capturing variation across respondents.

Binary Variables

Binary variables (e.g., is_migrant, has_child, urban_specialist, has_car, pet_dog) were summarized as simple Yes/No percentages. This highlighted imbalances such as car ownership (60%) or migrant status (27%).

Categorical Variables

Categorical variables (e.g., main_commute_type, residence_grp, parking_home, occupation) were summarized with frequency distributions. These distributions revealed, for example:

• Most respondents commute by motor vehicle (40%).
• Majority reside in Southern Europe (59%).
• Parking access is often split between free and no access.

1_Bivariate Analysis (R Script)

[BivariateAnalysis.R](1_Bivariate Analysis/BivariateAnalysis.R) performs all pairwise bivariate analyses within blocks and across domains, producing plots, correlation matrices, and a consolidated Excel workbook.

Design (Segmentation + Colour):

• Method: Pearson correlation (continuous)
• Outputs: Heatmap + matrix with significance flags
• Folder: Results_Bivariate/Design

Emotion/Perception Variables:

• Method: Polychoric correlation (ordinal/discrete)
• Outputs: Heatmap + correlation matrix
• Folder: Results_Bivariate/Emotion

Demographics:

• Method: Cramér's V (Categorical)
• Outputs: Heatmap + matrix with flags
• Folder: Results_Bivariate/Demographics

Cross-Domain:

All cross-domain results are under Results_Bivariate/Cross_Domains:

Analysis	Method	Output
Design by Emotion	Spearman correlation	Heatmap and matrix with flags
Design by Preference	Boxplot and Violin	Plots for each variable
Emotion by Preference	Mosaic plot	PNG and workbook cross-tab
Emotion by Demographics	Mosaic plot	PNG and workbook cross-tab
Demographics by Preference	Mosaic plot	PNG and workbook cross-tab

Excel Workbooks

Additionally, raw values and the methodology used for each pair of variables are summarized in the following excel workbooks:

• [Full_Bivariate_Analysis.xlsx](1_Bivariate Analysis/Results_Bivariate/Full_Bivariate_Analysis.xlsx) - All correlation/mosaic matrices with flags
• [Methods_Log.xlsx](1_Bivariate Analysis/Results_Bivariate/Methods_Log.xlsx) - Documentation of each computed variable pair, method, and plot type

1_Polychoric EFA and CFA on Perceptions

Perceptions_EFA_CFA - polychoric.R conducts an exploratory and confirmatory factor analysis of perception-related survey items (usefulness, safety, security, stress, comfort, order, liveability). Because all items were measured on ordinal Likert scales, analyses are based on polychoric correlations rather than Pearson correlations.

Checks

• Compute polychoric correlation matrix for ordinal items.
• Assess factorability using Kaiser-Meyer-Olkin (KMO) measure and Bartlett's test of sphericity.
• Perform parallel analysis to suggest the number of factors; results saved as scree_plot_FAonly.png.

Exploratory Factor Analysis (EFA)

• Tested 2-factor and 3-factor solutions using principal axis factoring with oblimin rotation.
• Outputs include factor loadings, communalities, eigenvalues, variance explained, and factor correlations.
• Results saved in separate Excel files: EFA_2F_results.xlsx and EFA_3F_results.xlsx.

Confirmatory Factor Analysis (CFA)

• 3-factor model specified based on EFA results:
  • Functionality: usefulness, order
  • Vulnerability: stress, safety, security
  • Habitability: comfort, liveability
• Inter-factor correlations allowed.
• Model estimated using WLSMV, robust for ordinal indicators.
• Outputs include:
  • Parameter estimates
  • Standardized loadings
  • Factor scores
  • Fit measures (chi-square, df, p-value, CFI, TLI, RMSEA, SRMR)
• All CFA outputs were saved in a single Excel file: CFA_3F_Results.xlsx.
• Factor score pairs plotted in CFA_3F_scores_pairs.png.

Fit Measures

• Evaluated using indices suitable for ordinal CFA:
  • CFI/TLI ≥ 0.95
  • RMSEA ≤ 0.06
  • SRMR ≤ 0.08
  • p-value ≥ 0.05
• AIC/BIC are not computed under WLSMV (not meaningful for ordinal indicators).

Results

• The final 3-factor CFA solution aligns with theoretical constructs and EFA results.

2_SEM - Training:

[SEM - term by term and base models.R](2_SEM Analysis/SEM - term by term and base models.R) performs structural equation modeling (SEM) using segmentation, color, and perception-related predictors. It also explores moderation effects of demographic variables on latent factors and preference outcomes.

1. Load Data & Categorize Variables

• Raw data and variable categorization sheets loaded.
• Variables separated into segmentation, color, emotion, continuous, categorical, binary, discrete.

2. Continuous Variable Handling

• Standardize continuous variables (z-scores).
• Keep CFA latent scores separate as they are not directly used in SEM.

3. Categorical Variable Handling

• Convert binary variables to factors.
• Identify multi-level demographic variables for dummy coding.

4. PCA for Color Variables

• Reduce brightness and saturation variables to 2 principal components:
  • vividness_saturation_sc
  • contrast_brightness_sc
• Factor scores added to dataset for SEM.

5. Measurement Model Definition

• Latent factors based on perception constructs:
  • Functionality = order + usefulness
  • Vulnerability = safety + security + stress
  • Habitability = comfort + liveability
• Latent factors allowed to correlate.

6. SEM Model Specification

• Segmentation SEM: all segmentation predictors to latent factors and preference.
• Colour SEM: all color predictors to latent factors and preference.
• Estimator: WLSMV (robust for ordinal dependent variable preference).

7. Fit SEM Models and Extract Results

• Standardized loadings, fit measures, modification indices, R².
• Save results to Excel (validation and for_tool folders).

8. Demographic Variable Binning & Dummy Coding

• Age, commute time, commute mode count grouped.
• Multi-level demographic variables dummied, with first category dropped.

9. Moderated SEM Analysis

• Residual-centered interaction terms: predictor x demographic moderator.
• Loops run three types of moderation:
  1. Predictor x Moderator to Latent Factors
  2. Predictor x Moderator to Preference
  3. Latent Factor x Moderator to Preference
• Significant interactions flagged (p < 0.05).
• Results saved separately for segmentation and colour predictors.

10. Final Results

- PCA outputs, gender & occupation mapping rules.  
- Moderation results (all & significant-only) exported for tool and validation.  
- Latent factor covariances extracted.  
- Means and SDs of continuous variables saved.  
- Residual-centering coefficients documented for reproducibility.

Notes / Example Insights

• Moderation loops allow systematic investigation of how demographics alter relationships between predictors, latent perceptions, and preference.
• PCA on colour variables reduces dimensionality while preserving key variance in brightness and saturation.
• Segmentation and colour predictors analyzed separately:
  • Avoids confounding effects and simplifies interpretation.
  • Each model focuses on conceptually distinct predictor sets: segmentation features (e.g., street elements) vs colour features (brightness/saturation of the environment).
• Moderation loops run term-by-term:
  • Computational feasibility: the number of observations isn't small but including all interactions at once would drastically increase model complexity.
  • Interpretability: term-by-term analysis allows clear identification of which specific predictor to demographic interactions affect latent factors or preference.
• Standardized SEM output provides effect sizes, significance, and model fit metrics for further interpretation.

3_Predictive Tool (SEM-based Scoring)

Prediction Tool - R Script predicts Preference probabilities for a holdout set using coefficients learned from the SEM training step. It builds two parallel predictors:

• Segmentation model (uses spatial/behavioral predictors)
• Colour model (uses colour/behavioral predictors)

Pre-processing

1. Split: Filter the test set by pair_seen == "201" (holdout).
2. Scaling: Recreate _sc variables using training means/SDs from continuous_means_sds.csv.
3. PCA factors: Reconstruct colour factors on the holdout by multiplying the scaled inputs
   (mean_brightness_var_sc, std_dev_brightness_var_sc, mean_saturation_var_sc, std_dev_saturation_var_sc)
   by the saved loadings to get:
   • vividness_saturation_sc
   • contrast_brightness_sc
4. Demographic recodes (exactly as trained):
   • gender: F = 1, M = 0, O = NA (rows with NA are dropped).
   • occupation: Employed = 1, Student/Retired-Unemployed = 0.
5. Binning (for alignment with moderation rules):
   • age_group: Young (≤29), Middle (30-49), Older (≥50)
   • commute_time_group: No commute / Short (1-2) / Long (3-4)
   • commute_mode_count_group: None / Single-mode / Two-modes / Multi-mode
6. Dummies: Create one-hot dummies for multi-level demographics to match the names that appeared in the significant moderation terms.
7. Column hygiene: Non-alphanumeric characters in column names are replaced with _.

Coefficient selection

From the base parameter tables:

• Latent targets (Functionality, Habitability, Vulnerability): keep paths with p < 0.05 or whose predictors appear in significant moderation to those latents.
• Preference target: keep direct paths with p < 0.05 or paths involving predictors that appear in significant moderation to Preference.
• Threshold: the probit threshold t1 is read from the Preference row where rhs == "t1".

From the moderation tables:

• Keeps only significant interaction terms (p < 0.05).
• Tags interactions by their target (one of the 3 latent factors or Preference).

A global shrink factor (default 1) can be applied to all non-threshold estimates for calibration.

Interaction reconstruction (residual-centering)

Significant interaction terms are recomputed in the holdout using residual-centering with the stored OLS coefficients in
resid_coefs_seg.csv:

• Build raw product pred * moderator.
• Subtract fitted values from lm(pred*moderator ~ pred + moderator) using the saved (Intercept), pred, and moderator coefficients.
• This reproduces the exact interaction scale used during training.

Prediction pipeline

1. Predict latent factor scores
   For each model (Segmentation and Colour) and for each latent L in {Functionality, Habitability, Vulnerability}:
   Latent_L = SUM(base_coef_Lj * Xj) + SUM(mod_coef_Lk * Interaction_k)

2. Predict Preference (probit link)
   Let the linear predictor be:
   LP = SUM(base_coef * [Latent_factors and/or direct predictors]) + SUM(mod_coef * Interaction)
   Then the probability of Preference = 1 is:
   P = Phi(LP - t1) * 100
   where Phi is the standard normal CDF (probit). P is reported in percent.

Notes & assumptions

• Separate models: Segmentation and Colour models are kept separate to mirror training (different feature spaces, clearer interpretation, and avoids overfitting by pooling many unrelated predictors).
• Term-by-term moderation: Interactions are tested and carried forward individually for interpretability and because the full set of interactions is large relative to sample size. This avoids unstable joint estimation and highlights the most practically relevant effects.
• Calibration: shrink_factor (default 1) can be tuned to temper estimates for external validity.
• Deterministic scoring: All preprocessing (scaling, PCA, coding, residual-centering) is done using saved artifacts from training to ensure strict train/test separation.
• Probit threshold: t1 is read from the fitted ordinal link (Preference). If absent, the script defaults to 0.

4_Master Script (Holdout Automation):

[4 - SEM_MasterScript.R](4 - SEM_MasterScript.R) automates train/test loops across multiple or singular holdout groups.

Holdout groups are predefined in holdout_groups (e.g., "101", "201", "301","405", etc.).
Certain groups are excluded because removing them takes away too much information for stable SEM estimation.

• Backup original scripts (SEM - term by term and base models.R and SEM - Predictive Tool.R) before modification.
  For each group:
  • Update the train script with the current holdout_grp.
  • Run training via Rscript (SEM estimation).
  • Update the test script with the same holdout.
  • Run testing via Rscript (predictive scoring).
  • Move results from:
    • 2_SEM Analysis to 2_SEM Analysis/holdout_X
    • 3_Predictive Tool to 3_Predictive Tool/holdout_X
      where X is the iteration number.
  • Exclude .R files and existing holdout_ folders from being moved.
    After all holdouts finish, restore the original training and testing scripts.

Notes:

• This wrapper provides strict train/test separation by dynamically swapping the holdout group in both scripts.
• Prevents overwriting results by automatically creating holdout_X folders.
• Stops execution if either training or testing fails for a group, ensuring reproducibility.

4_Prediction tunning:

To facilitate the prediction of different groups over the same training estimates without re-running the training script, [4 - SEM_MasterScript - Predict Only.R](4 - SEM_MasterScript - Predict Only.R) was created. To evaluate those sets of predictions, [4 - Summarise results.py](4 - Summarise results.py) was coded. After running [4 - Summarise results.py](4 - Summarise results.py) a simple table was made inside each output excel file. To make the comparison of different sets of predictions easier, [Shrink factors tunning.xlsx](Shrink factors tunning.xlsx) references all result files in one sheet.

5_Prediction Use Case:

To demonstrate the methodology's use, 5 interventions were created from one street image. These were then passed through colour and segmentation analysis to get their comparison data. Afterwards, the same survey data was used in conjunction with the new image-based data to predict the preference probabilities of all interventions. To allow for easier analysis of every intervention, [Predictions - Summary.xlsx](5_Prediction Use Case/Predictions/Predictions - Summary.xlsx) was created by merging all results. After computing min-max observation scores per intervention, a pivot table was created for both colour and segmentation systems to enable not only an overall analysis of the 5 transformations, but also a deeper analysis by group of demographic or habit characteristics.