|
226 | 226 | "from parcels import AdvectionRK4, FieldSet, JITParticle, ParticleSet\n", |
227 | 227 | "\n", |
228 | 228 | "\n", |
229 | | - "def set_cmems_fieldset(cs):\n", |
| 229 | + "def set_cms_fieldset(cs):\n", |
230 | 230 | " data_dir_head = \"/data/oceanparcels/input_data\"\n", |
231 | | - " data_dir = os.path.join(data_dir_head, \"CMEMS/GLOBAL_REANALYSIS_PHY_001_030/\")\n", |
| 231 | + " data_dir = os.path.join(data_dir_head, \"CMS/GLOBAL_REANALYSIS_PHY_001_030/\")\n", |
232 | 232 | " files = sorted(glob(data_dir + \"mercatorglorys12v1_gl12_mean_201607*.nc\"))\n", |
233 | 233 | " variables = {\"U\": \"uo\", \"V\": \"vo\"}\n", |
234 | 234 | " dimensions = {\"lon\": \"longitude\", \"lat\": \"latitude\", \"time\": \"time\"}\n", |
|
242 | 242 | "mem_used_GB = []\n", |
243 | 243 | "chunksize = [128, 256, 512, 768, 1024, 1280, 1536, 1792, 2048, 2610, \"auto\", False]\n", |
244 | 244 | "for cs in chunksize:\n", |
245 | | - " fieldset = set_cmems_fieldset(cs)\n", |
| 245 | + " fieldset = set_cms_fieldset(cs)\n", |
246 | 246 | " pset = ParticleSet(\n", |
247 | 247 | " fieldset=fieldset, pclass=JITParticle, lon=[0], lat=[0], repeatdt=delta(hours=1)\n", |
248 | 248 | " )\n", |
|
342 | 342 | "\n", |
343 | 343 | "It may - depending on your simulation goal - be necessary to tweak the chunksize to leave more memory space for additional particles that are being simulated. Since particles and fields share the same memory space, lower memory utilisation by the `FieldSet` means more memory available for a larger `ParticleSet`.\n", |
344 | 344 | "\n", |
345 | | - "Also note that the above example is for a 2D application. For 3D applications, the `chunksize=False` will almost always be slower than `chunksize='auto'` or any dictionary, and is likely to run into insufficient memory issues, raising a `MemoryError`. The plot below shows the same analysis as above, but this time for a set of simulations using the full 3D CMEMS code. In this case, the `chunksize='auto'` is about two orders of magnitude faster than running without chunking, and about 7.5 times faster than with minimal chunk capacity (i.e. `chunksize=(1, 128, 128)`).\n", |
| 345 | + "Also note that the above example is for a 2D application. For 3D applications, the `chunksize=False` will almost always be slower than `chunksize='auto'` or any dictionary, and is likely to run into insufficient memory issues, raising a `MemoryError`. The plot below shows the same analysis as above, but this time for a set of simulations using the full 3D Copernicus Marine Service code. In this case, the `chunksize='auto'` is about two orders of magnitude faster than running without chunking, and about 7.5 times faster than with minimal chunk capacity (i.e. `chunksize=(1, 128, 128)`).\n", |
346 | 346 | "\n", |
347 | 347 | "Choosing too small chunksizes can make the code slower, again highlighting that it is wise to explore which chunksize is best for your experiment before you perform it.\n" |
348 | 348 | ] |
|
368 | 368 | "from parcels import AdvectionRK4_3D\n", |
369 | 369 | "\n", |
370 | 370 | "\n", |
371 | | - "def set_cmems_fieldset_3D(cs):\n", |
| 371 | + "def set_cms_fieldset_3D(cs):\n", |
372 | 372 | " data_dir_head = \"/data/oceanparcels/input_data\"\n", |
373 | | - " data_dir = os.path.join(data_dir_head, \"CMEMS/GLOBAL_REANALYSIS_PHY_001_030/\")\n", |
| 373 | + " data_dir = os.path.join(data_dir_head, \"CMS/GLOBAL_REANALYSIS_PHY_001_030/\")\n", |
374 | 374 | " files = sorted(glob(data_dir + \"mercatorglorys12v1_gl12_mean_201607*.nc\"))\n", |
375 | 375 | " variables = {\"U\": \"uo\", \"V\": \"vo\"}\n", |
376 | 376 | " dimensions = {\n", |
|
393 | 393 | "chunksize_3D = [128, 256, 512, 768, 1024, 1280, 1536, 1792, 2048, 2610, \"auto\", False]\n", |
394 | 394 | "func_time3D = []\n", |
395 | 395 | "for cs in chunksize_3D:\n", |
396 | | - " fieldset = set_cmems_fieldset_3D(cs)\n", |
| 396 | + " fieldset = set_cms_fieldset_3D(cs)\n", |
397 | 397 | " pset = ParticleSet(\n", |
398 | 398 | " fieldset=fieldset, pclass=JITParticle, lon=[0], lat=[0], repeatdt=delta(hours=1)\n", |
399 | 399 | " )\n", |
|
0 commit comments