Skip to content

Commit 960bdc1

Browse files
committed
Merge branch 'fix/deallocate-missing-arrays' of https://github.com/SVS87/MFC into fix/deallocate-missing-arrays
2 parents 80e6d0a + 1274926 commit 960bdc1

21 files changed

Lines changed: 793 additions & 803 deletions

docs/documentation/case.md

Lines changed: 28 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -320,7 +320,8 @@ This is enabled by adding ``'elliptic_smoothing': "T",`` and ``'elliptic_smoothi
320320
| `num_ibs` | Integer | Number of immersed boundary patches |
321321
| `num_stl_models` | Integer | Number of STL/OBJ model entries in the `stl_models` array |
322322
| `num_particle_clouds` | Integer | Number of particle bed specifications to generate immersed boundary patches from |
323-
| `ib_neighborhood_radius` | Integer | Parameter that controls the neighborhood size for IB detection. |
323+
| `ib_neighborhood_radius` | Integer | Parameter that controls the neighborhood size for IB detection. |
324+
| `many_ib_patch_parallelism` | Logical | Parallelize over IB patches instead of grid cells (better for many small patches). |
324325
| `geometry` | Integer | Geometry configuration of the patch.|
325326
| `x[y,z]_centroid` | Real | Centroid of the applied geometry in the [x,y,z]-direction. |
326327
| `length_x[y,z]` | Real | Length, if applicable, in the [x,y,z]-direction. |
@@ -368,7 +369,7 @@ Additional details on this specification can be found in [NACA airfoil](https://
368369

369370
- For STL/OBJ geometry (geometry 5 or 12), set `model_id` to index into the `stl_models` array and specify `model_filepath`, `model_scale`, `model_translate`, and `model_threshold` on that entry.
370371

371-
- `moving_ibm` sets the method by which movement will be applied to the immersed boundary. Using 0 will result in no movement. Using 1 will result 1-way coupling where the boundary moves at a constant rate and applied forces to the fluid based upon it's own motion. In 1-way coupling, the fluid does not apply forces back onto the IB. Using 2 will result in 2-way coupling, where the boundary pushes on the fluid and the fluid pushes back on the boundary via pressure and viscous forces. If external forces are applied, the boundary will also experience those forces.
372+
- `moving_ibm` sets the method by which movement will be applied to the immersed boundary. Using 0 will result in no movement. Using 1 will result 1-way coupling where the boundary moves at a constant rate and applied forces to the fluid based upon its own motion. In 1-way coupling, the fluid does not apply forces back onto the IB. Using 2 will result in 2-way coupling, where the boundary pushes on the fluid and the fluid pushes back on the boundary via pressure and viscous forces. If external forces are applied, the boundary will also experience those forces.
372373

373374
- `vel(i)` is the initial linear velocity of the IB in the x, y, z direction for i=1, 2, 3. When `moving_ibm` equals 2, this velocity is just the starting speed of the object, which will then accelerate due to external forces. If `moving_ibm` equals 1, then this is constant if it is a number, or can be described analytically with an expression.
374375

@@ -378,15 +379,35 @@ Additional details on this specification can be found in [NACA airfoil](https://
378379
Available variables: `x` (`x_cc(i)`), `y` (`y_cc(j)`), `z` (`z_cc(k)`), `t` (current simulation time), and `r` (the IB patch radius).
379380
The same intrinsic functions and `pi` constant apply; bare `e` is not available.
380381

381-
- `coefficient_of_restitution` is a number from 0 (exclusive) to 1 (inclusive) describing how elastic IB collisions are. 0 is for perfectly inellastic collisions while 1 is for perfectly ellastic collisions.
382+
- `coefficient_of_restitution` is a number from 0 (exclusive) to 1 (inclusive) describing how elastic IB collisions are. 0 is for perfectly inelastic collisions while 1 is for perfectly elastic collisions.
382383

383-
- `collision_model` is an integer to select the collision model being used for IB collisions. Using 0 disables collisions and collisiono checking. 1 enables the soft-sphere collision model, where all IBs must be circles or sphere and those IBs can collide with each other as well as walls.
384+
- `collision_model` is an integer to select the collision model being used for IB collisions. Using 0 disables collisions and collision checking. 1 enables the soft-sphere collision model, where all IBs must be circles or sphere and those IBs can collide with each other as well as walls.
384385

385-
- `collision_time` is approximately the amount of simulation time used to resolve collisions. This is handled by modifying the spring gonstant used to apply collision forces.
386+
- `collision_time` is approximately the amount of simulation time used to resolve collisions. This is handled by modifying the spring constant used to apply collision forces.
386387

387388
- `ib_coefficient_of_friction` is the coefficient of friction used in IB collisions.
388389

389-
- `ib_neighborhood_radius` controls the size of the neighborhood size. This value defaults to 1, which indicates that any given rank is aware of IB's up to 1 ranks away. This parameter is required to strong-scale a case when IB's eventually grow to be larger than one full processor domain wide.
390+
- `ib_neighborhood_radius` controls the size of the neighborhood size. This value defaults to 1, which indicates that any given rank is aware of IBs up to 1 ranks away. This parameter is required to strong-scale a case when IBs eventually grow to be larger than one full processor domain wide.
391+
392+
#### Particle Clouds
393+
394+
A particle cloud is a compact specification of a bed of identical circular (2D) or spherical (3D) immersed boundaries; each cloud is expanded into individual `patch_ib` particles at startup. Set `num_particle_clouds` to the number of beds and prepend the parameters below with `particle_cloud(j)%` where $j$ is the cloud index.
395+
396+
| Parameter | Type | Description |
397+
| ---: | :----: | :--- |
398+
| `x[y,z]_centroid` | Real | Centre of the cloud region in the [x,y,z]-direction. |
399+
| `length_x[y,z]` | Real | Extent of the cloud region in the [x,y,z]-direction. |
400+
| `num_particles` | Integer | Number of particles to place in the region. |
401+
| `radius` | Real | Radius of every particle in the cloud. |
402+
| `mass` | Real | Mass of every particle in the cloud. |
403+
| `min_spacing` | Real | Minimum surface-to-surface gap between particles (centres are `2*radius + min_spacing` apart). |
404+
| `moving_ibm` | Integer | Motion flag applied to every particle (see `patch_ib(j)%%moving_ibm`). |
405+
| `seed` | Integer | Random seed for reproducible placement (used by `packing_method = 1`). |
406+
| `packing_method` | Integer | Algorithm used to place the particles. |
407+
408+
- `packing_method` selects how the `num_particles` are positioned within the cloud region:
409+
- `1` (rejection sampling) draws random positions and rejects any that violate `min_spacing`, producing a disordered bed. `seed` makes the placement reproducible.
410+
- `2` (lattice) places the particles on the optimally dense lattice for the geometry — a triangular lattice in 2D and a face-centered cubic lattice in 3D. The lattice spacing is derived from the particle density (`num_particles` over the region area/volume); if that spacing is below the required `2*radius + min_spacing`, the region is too dense and the run aborts.
390411

391412
### 5. Fluid Material's {#sec-fluid-materials}
392413

@@ -1248,7 +1269,7 @@ Boundary is at polar angle \f$\theta = \mathrm{atan2}(y - y_{\mathrm{centroid}},
12481269
| 3 | 2D Rectangle | 2 |
12491270
| 4 | 2D Airfoil | 2 |
12501271
| 8 | 3D Sphere | 3 |
1251-
| 10 | 3D Cylinder | 3 |
1272+
| 10 | 3D Cylinder | 3 | `length_x` sets the axial length of the cylinder. |
12521273
| 11 | 3D Airfoil | 3 |
12531274

12541275
### Acoustic Supports {#acoustic-supports}

docs/index.html

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@
6363
{ name: "Burstwave lithotripsy", image: "res/simulations/k.png", computer: "Delta", computerUrl: "https://www.ncsa.illinois.edu/research/project-highlights/delta/", accelerators: "128 A100s", walltime: "30m", source: "https://www.youtube.com/watch?v=XWsUTaJXGF8" },
6464
{ name: "Cavitation fragments kidney stone", image: "res/simulations/d.png", computer: "Summit", computerUrl: "https://www.olcf.ornl.gov/summit/", accelerators: "576 V100s", walltime: "30m", source: "https://doi.org/10.48550/arXiv.2305.09163" },
6565
{ name: "Focused ultrasound on kidney stone", image: "res/simulations/v.png", computer: "Bridges2", computerUrl: "https://www.psc.edu/resources/bridges-2/", accelerators: "10 V100s", walltime: "2h", source: "https://www.youtube.com/watch?v=z8j3NH-Y6i0" },
66+
{ name: "Microbubble-enhanced HIFU in a liver", image: "res/simulations/y.png", computer: "Delta", computerUrl: "https://www.ncsa.illinois.edu/research/project-highlights/delta/", accelerators: "128 CPU cores", walltime: "26m", source: "https://www.youtube.com/watch?v=TbR0MEdG8OU" },
6667
{ name: "Kidney stone stress waves", image: "res/simulations/l.png", computer: "Bridges2", computerUrl: "https://www.psc.edu/resources/bridges-2/", accelerators: "8 V100s", walltime: "20m", source: "https://www.youtube.com/watch?v=Q2L0J68qnRw" },
6768
{ name: "Whale bubble net feeding", image: "res/simulations/p.png", computer: "Delta", computerUrl: "https://www.ncsa.illinois.edu/research/project-highlights/delta/", accelerators: "128 A100s", walltime: "30m", source: "https://www.youtube.com/watch?v=6EpP6tdCZSA" },
6869
{ name: "Earplug acoustics (kinetic energy)", image: "res/simulations/q.png", computer: "Delta", computerUrl: "https://www.ncsa.illinois.edu/research/project-highlights/delta/", accelerators: "8 A100s", walltime: "5h", source: "https://www.youtube.com/watch?v=xSW5wZkdbrc" },

docs/module_categories.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,8 @@
8181
"m_checker",
8282
"m_checker_common",
8383
"m_sim_helpers",
84-
"m_derived_variables"
84+
"m_derived_variables",
85+
"m_patch_geometries"
8586
]
8687
},
8788
{

docs/res/simulations/y.png

169 KB
Loading

examples/2D_mibm_particle_cloud/case.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@
7979
"ib": "T",
8080
"num_ibs": 0,
8181
"viscous": "T",
82+
"many_ib_patch_parallelism": "T",
8283
# Collision model (soft-sphere, from 3D_mibm_sphere_head_on_collision)
8384
"collision_model": 1,
8485
"coefficient_of_restitution": 0.9,

src/common/m_derived_types.fpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -372,7 +372,7 @@ module m_derived_types
372372
real(wp) :: min_spacing !< Minimum surface-to-surface gap (particle centers are 2*radius + min_spacing apart)
373373
integer :: moving_ibm !< Motion flag: 0=static, 1=moving (forces), 2=forced path
374374
integer :: seed !< Random seed for reproducible placement
375-
integer :: packing_method !< Packing algorithm: 1=rejection sampling
375+
integer :: packing_method !< Packing algorithm: 1=rejection sampling, 2=lattice
376376
end type particle_cloud_parameters
377377

378378
!> Derived type annexing the physical parameters (PP) of the fluids. These include the specific heat ratio function and liquid

src/common/m_model.fpp

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ module m_model
3535
real(wp), allocatable, dimension(:,:,:,:) :: gpu_boundary_v
3636
integer, allocatable :: gpu_boundary_edge_count(:)
3737
real(wp), allocatable :: stl_bounding_boxes(:,:,:)
38-
$:GPU_DECLARE(create='[gpu_ntrs, gpu_trs_v, gpu_trs_n, gpu_boundary_v, gpu_boundary_edge_count]')
38+
$:GPU_DECLARE(create='[gpu_ntrs, gpu_trs_v, gpu_trs_n, gpu_boundary_v, gpu_boundary_edge_count, stl_bounding_boxes]')
3939

4040
contains
4141

@@ -877,7 +877,7 @@ contains
877877
#endif
878878
if (num_stl_models == 0) return
879879

880-
allocate (stl_bounding_boxes(num_stl_models,1:3,1:3))
880+
@:ALLOCATE(stl_bounding_boxes(num_stl_models,1:3,1:3))
881881
@:ALLOCATE(models(num_stl_models))
882882

883883
do stl_id = 1, num_stl_models
@@ -950,6 +950,9 @@ contains
950950
end if
951951
end do
952952

953+
$:GPU_UPDATE(device='[stl_bounding_boxes]')
954+
$:GPU_UPDATE(device='[stl_models(1:num_stl_models)]')
955+
953956
! Pack and upload flat arrays for GPU (AFTER the loop)
954957
block
955958
integer :: mid, max_ntrs

src/common/m_patch_geometries.fpp

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
!>
2+
!! @file
3+
!! @brief Contains module m_ibm
4+
5+
#:include 'macros.fpp'
6+
7+
!> @brief Contains helper functions specific to various patch gemoetries for determining if a grid cell lies inside of or outside of
8+
!! a patch geometry
9+
module m_patch_geometries
10+
11+
use m_derived_types
12+
use m_global_parameters
13+
use m_variables_conversion
14+
use m_helper
15+
use m_helper_basic
16+
use m_constants
17+
use m_model
18+
19+
implicit none
20+
21+
public :: f_is_inside_sphere, f_is_inside_cylinder, f_is_inside_cuboid, f_is_inside_airfoil, f_is_inside_ellipse
22+
23+
contains
24+
25+
!> Check if the x, y, and z coordinates would be located inside a sphere with the patch_id's radius
26+
function f_is_inside_sphere(x, y, z, radius) result(is_inside)
27+
28+
$:GPU_ROUTINE(parallelism='[seq]')
29+
30+
real(wp), intent(in) :: radius, x, y, z
31+
logical :: is_inside
32+
33+
is_inside = x**2 + y**2 + z**2 <= radius**2
34+
35+
end function f_is_inside_sphere
36+
37+
!> Check which length of the cylinder is not default. Use that direction as the height and the other two coordinate
38+
! values as the radius check
39+
function f_is_inside_cylinder(polar_x, polar_y, height, radius, length) result(is_inside)
40+
41+
$:GPU_ROUTINE(parallelism='[seq]')
42+
43+
real(wp), intent(in) :: polar_x, polar_y, height, radius, length
44+
logical :: is_inside
45+
46+
! check if the circular component of the cylinder is correct
47+
is_inside = polar_x**2 + polar_y**2 <= radius**2
48+
49+
! in 3D, also check the length of the cylinder
50+
if (num_dims == 3) is_inside = is_inside .and. -0.5_wp*length <= height .and. 0.5_wp*length >= height
51+
52+
end function f_is_inside_cylinder
53+
54+
!> Check if the x, y, and possibly z coordinates would be located inside a cuboid with the patch_id's lengths
55+
function f_is_inside_cuboid(x, y, z, length) result(is_inside)
56+
57+
$:GPU_ROUTINE(parallelism='[seq]')
58+
59+
real(wp), intent(in) :: x, y, z
60+
real(wp), dimension(3), intent(in) :: length
61+
logical :: is_inside
62+
63+
! check if x and y are inside the rectangle plane at z=0
64+
is_inside = -0.5_wp*length(1) <= x .and. 0.5_wp*length(1) >= x .and. -0.5_wp*length(2) <= y .and. 0.5_wp*length(2) >= y
65+
66+
! if we are in 3D, this is a cuboid and so we must also check the z axis
67+
if (num_dims == 3) is_inside = is_inside .and. -0.5_wp*length(3) <= z .and. 0.5_wp*length(3) >= z
68+
69+
end function f_is_inside_cuboid
70+
71+
!> Check if the x, y, are bounded by a NACA airfoil. Check if the z coordinate is inside the left and right edges of the
72+
!! airfoil, if set.
73+
function f_is_inside_airfoil(x, y, z, length, airfoil_id) result(is_inside)
74+
75+
$:GPU_ROUTINE(parallelism='[seq]')
76+
77+
real(wp), intent(in) :: x, y, z, length
78+
integer, intent(in) :: airfoil_id
79+
logical :: is_inside
80+
integer :: k
81+
real(wp) :: f
82+
83+
is_inside = .false.
84+
85+
! check the initial x bounds of the grid cell
86+
if (.not. (x >= 0._wp .and. x <= ib_airfoil(airfoil_id)%c)) return
87+
88+
! if we are in 3D, we must also check the z axis
89+
if (num_dims == 3 .and. (.not. (-0.5_wp*length <= z .and. 0.5_wp*length >= z))) return
90+
91+
! our check branches for the upper and lower half of the airfoil
92+
if (y >= 0._wp) then
93+
! increment the iterator so we know where in the airfoil arrays to look
94+
k = 1
95+
do while (ib_airfoil_grids(airfoil_id)%upper(k)%x < x)
96+
k = k + 1
97+
end do
98+
99+
! If the values are approximately equivalent, skip the next check
100+
if (f_approx_equal(ib_airfoil_grids(airfoil_id)%upper(k)%x, x)) then
101+
if (y <= ib_airfoil_grids(airfoil_id)%upper(k)%y) is_inside = .true.
102+
else
103+
! check if the y value is below the upper edge of the airfoil
104+
f = (ib_airfoil_grids(airfoil_id)%upper(k)%x - x)/(ib_airfoil_grids(airfoil_id)%upper(k)%x &
105+
& - ib_airfoil_grids(airfoil_id)%upper(k - 1)%x)
106+
if (y <= ((1._wp - f)*ib_airfoil_grids(airfoil_id)%upper(k)%y + f*ib_airfoil_grids(airfoil_id)%upper(k - 1)%y)) &
107+
& is_inside = .true.
108+
end if
109+
else
110+
! increment the iterator so we know where in the airfoil arrays to look
111+
k = 1
112+
do while (ib_airfoil_grids(airfoil_id)%lower(k)%x < x)
113+
k = k + 1
114+
end do
115+
116+
! If the values are approximately equivalent, skip the next check
117+
if (f_approx_equal(ib_airfoil_grids(airfoil_id)%lower(k)%x, x)) then
118+
if (y >= ib_airfoil_grids(airfoil_id)%lower(k)%y) is_inside = .true.
119+
else
120+
! check if the y value is above the lower edge of the airfoil
121+
f = (ib_airfoil_grids(airfoil_id)%lower(k)%x - x)/(ib_airfoil_grids(airfoil_id)%lower(k)%x &
122+
& - ib_airfoil_grids(airfoil_id)%lower(k - 1)%x)
123+
if (y >= ((1._wp - f)*ib_airfoil_grids(airfoil_id)%lower(k)%y + f*ib_airfoil_grids(airfoil_id)%lower(k - 1)%y)) &
124+
& is_inside = .true.
125+
end if
126+
end if
127+
128+
end function f_is_inside_airfoil
129+
130+
function f_is_inside_ellipse(x, y, length) result(is_inside)
131+
132+
$:GPU_ROUTINE(parallelism='[seq]')
133+
134+
real(wp), intent(in) :: x, y
135+
real(wp), dimension(3), intent(in) :: length
136+
logical :: is_inside
137+
138+
! Ellipse condition (x/a)^2 + (y/b)^2 <= 1
139+
is_inside = (x/(0.5_wp*length(1)))**2 + (y/(0.5_wp*length(2)))**2 <= 1._wp
140+
141+
end function f_is_inside_ellipse
142+
143+
end module m_patch_geometries

src/post_process/m_global_parameters.fpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,10 @@ module m_global_parameters
9797

9898
! shear_num, shear_indices, shear_BC_flip_num, shear_BC_flip_indices: in m_global_parameters_common
9999
! proc_coords, start_idx, mpiiofs, mpi_info_int: in m_global_parameters_common
100+
type(ib_airfoil_parameters), allocatable, dimension(:) :: ib_airfoil !< Per-airfoil NACA parameters (unused in post_process)
101+
!> Per-airfoil computed surface grids (unused in post_process)
102+
type(ib_airfoil_grid), allocatable, dimension(:) :: ib_airfoil_grids
103+
100104
#ifdef MFC_MPI
101105
type(mpi_io_var), public :: MPI_IO_DATA
102106
type(mpi_io_ib_var), public :: MPI_IO_IB_DATA

src/pre_process/m_global_parameters.fpp

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,12 @@ module m_global_parameters
7070
integer :: nmom !< Number of carried moments
7171
!> @}
7272

73+
!> @name Immersed Boundaries
74+
!> @{
7375
! patch_ib, ib_airfoil, stl_models: auto-generated in generated_decls.fpp
76+
!> Per-airfoil computed surface grids (unused in pre_process)
77+
type(ib_airfoil_grid), allocatable, dimension(:) :: ib_airfoil_grids
78+
!> @}
7479

7580
!> @name Non-polytropic bubble gas compression
7681
!> @{

0 commit comments

Comments
 (0)