Can't allocate zeros/ones on MI300A

[williamfgc]

Questionnaire

1. Does ROCm works for you outside of Julia, e.g. C/C++/Python?
   Yes

2. Post output of rocminfo.

ROCk module is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.18
Runtime Ext Version:     1.15
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
XNACK enabled:           NO
DMAbuf Support:          YES
VMM Support:             NO

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                    AMD Instinct MI300A Accelerator    
  Uuid:                    CPU-XX                             
  Marketing Name:          AMD Instinct MI300A Accelerator    
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    0                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      32768(0x8000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   3700                               
  BDFID:                   0                                  
  Internal Node ID:        0                                  
  Compute Unit:            48                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Memory Properties:       
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    131274592(0x7d31760) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    131274592(0x7d31760) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    131274592(0x7d31760) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 4                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    131274592(0x7d31760) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 2                  
*******                  
  Name:                    AMD Instinct MI300A Accelerator    
  Uuid:                    CPU-XX                             
  Marketing Name:          AMD Instinct MI300A Accelerator    
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    1                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      32768(0x8000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   3700                               
  BDFID:                   0                                  
  Internal Node ID:        1                                  
  Compute Unit:            48                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Memory Properties:       
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    131804804(0x7db2e84) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    131804804(0x7db2e84) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    131804804(0x7db2e84) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 4                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    131804804(0x7db2e84) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 3                  
*******                  
  Name:                    AMD Instinct MI300A Accelerator    
  Uuid:                    CPU-XX                             
  Marketing Name:          AMD Instinct MI300A Accelerator    
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    2                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      32768(0x8000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   3700                               
  BDFID:                   0                                  
  Internal Node ID:        2                                  
  Compute Unit:            48                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Memory Properties:       
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    131804804(0x7db2e84) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    131804804(0x7db2e84) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    131804804(0x7db2e84) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 4                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    131804804(0x7db2e84) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 4                  
*******                  
  Name:                    AMD Instinct MI300A Accelerator    
  Uuid:                    CPU-XX                             
  Marketing Name:          AMD Instinct MI300A Accelerator    
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    3                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      32768(0x8000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   3700                               
  BDFID:                   0                                  
  Internal Node ID:        3                                  
  Compute Unit:            48                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Memory Properties:       
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    131753576(0x7da6668) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    131753576(0x7da6668) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    131753576(0x7da6668) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 4                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    131753576(0x7da6668) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 5                  
*******                  
  Name:                    gfx942                             
  Uuid:                    GPU-a70564cccd47836c               
  Marketing Name:          AMD Instinct MI300A                
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    4                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      32(0x20) KB                        
    L2:                      4096(0x1000) KB                    
    L3:                      262144(0x40000) KB                 
  Chip ID:                 29856(0x74a0)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   2100                               
  BDFID:                   512                                
  Internal Node ID:        4                                  
  Compute Unit:            228                                
  SIMDs per CU:            4                                  
  Shader Engines:          24                                 
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    TRUE                               
  Memory Properties:       APU
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        2147483647(0x7fffffff)             
    y                        65535(0xffff)                      
    z                        65535(0xffff)                      
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 180                                
  SDMA engine uCode::      19                                 
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 4                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx942:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
    ISA 2                    
      Name:                    amdgcn-amd-amdhsa--gfx9-4-generic:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
*******                  
Agent 6                  
*******                  
  Name:                    gfx942                             
  Uuid:                    GPU-96ebf5ea650564c9               
  Marketing Name:          AMD Instinct MI300A                
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    5                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      32(0x20) KB                        
    L2:                      4096(0x1000) KB                    
    L3:                      262144(0x40000) KB                 
  Chip ID:                 29856(0x74a0)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   2100                               
  BDFID:                   16640                              
  Internal Node ID:        5                                  
  Compute Unit:            228                                
  SIMDs per CU:            4                                  
  Shader Engines:          24                                 
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    TRUE                               
  Memory Properties:       APU
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        2147483647(0x7fffffff)             
    y                        65535(0xffff)                      
    z                        65535(0xffff)                      
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 180                                
  SDMA engine uCode::      19                                 
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 4                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx942:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
    ISA 2                    
      Name:                    amdgcn-amd-amdhsa--gfx9-4-generic:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
*******                  
Agent 7                  
*******                  
  Name:                    gfx942                             
  Uuid:                    GPU-5b1027d694a42766               
  Marketing Name:          AMD Instinct MI300A                
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    6                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      32(0x20) KB                        
    L2:                      4096(0x1000) KB                    
    L3:                      262144(0x40000) KB                 
  Chip ID:                 29856(0x74a0)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   2100                               
  BDFID:                   33024                              
  Internal Node ID:        6                                  
  Compute Unit:            228                                
  SIMDs per CU:            4                                  
  Shader Engines:          24                                 
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    TRUE                               
  Memory Properties:       APU
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        2147483647(0x7fffffff)             
    y                        65535(0xffff)                      
    z                        65535(0xffff)                      
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 180                                
  SDMA engine uCode::      19                                 
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 4                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx942:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
    ISA 2                    
      Name:                    amdgcn-amd-amdhsa--gfx9-4-generic:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
*******                  
Agent 8                  
*******                  
  Name:                    gfx942                             
  Uuid:                    GPU-c863b92cbe151628               
  Marketing Name:          AMD Instinct MI300A                
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    7                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      32(0x20) KB                        
    L2:                      4096(0x1000) KB                    
    L3:                      262144(0x40000) KB                 
  Chip ID:                 29856(0x74a0)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   2100                               
  BDFID:                   49408                              
  Internal Node ID:        7                                  
  Compute Unit:            228                                
  SIMDs per CU:            4                                  
  Shader Engines:          24                                 
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    TRUE                               
  Memory Properties:       APU
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        2147483647(0x7fffffff)             
    y                        65535(0xffff)                      
    z                        65535(0xffff)                      
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 180                                
  SDMA engine uCode::      19                                 
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    65829720(0x3ec7b58) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 4                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx942:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
    ISA 2                    
      Name:                    amdgcn-amd-amdhsa--gfx9-4-generic:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)             
        y                        65535(0xffff)                      
        z                        65535(0xffff)                      
      FBarrier Max Size:       32                                 
*** Done ***             

3. Post output of AMDGPU.versioninfo() if possible.

julia> AMDGPU.versioninfo()
AMDGPU versioninfo
┌───────────┬──────────────────┬───────────┬─────────────────────────────────────────────────────────────────────
│ Available │ Name             │ Version   │ Path                                                               ⋯
├───────────┼──────────────────┼───────────┼─────────────────────────────────────────────────────────────────────
│     +     │ LLD              │ -         │ /opt/rocm-7.2.0/lib/llvm/bin/ld.lld                                ⋯
│     +     │ Device Libraries │ -         │ /home/wfg/.julia-1.12-faraday/artifacts/0a0cad46b7dacc66387ed19e8c ⋯
│     +     │ HIP              │ 7.2.26015 │ /opt/rocm-7.2.0/lib/libamdhip64.so                                 ⋯
│     +     │ rocBLAS          │ 5.2.0     │ /opt/rocm-7.2.0/lib/librocblas.so                                  ⋯
│     +     │ rocSOLVER        │ 3.32.0    │ /opt/rocm-7.2.0/lib/librocsolver.so                                ⋯
│     +     │ rocSPARSE        │ 4.2.0     │ /opt/rocm-7.2.0/lib/librocsparse.so                                ⋯
│     +     │ rocRAND          │ 4.2.0     │ /opt/rocm-7.2.0/lib/librocrand.so                                  ⋯
│     +     │ rocFFT           │ 1.0.36    │ /opt/rocm-7.2.0/lib/librocfft.so                                   ⋯
│     +     │ MIOpen           │ 3.5.1     │ /opt/rocm-7.2.0/lib/libMIOpen.so                                   ⋯
└───────────┴──────────────────┴───────────┴─────────────────────────────────────────────────────────────────────
                                                                                                 1 column omitted

AMDGPU devices
┌────┬─────────────────────┬────────────────────────┬───────────┬────────────┬───────────────┐
│ Id │                Name │               GCN arch │ Wavefront │     Memory │ Shared Memory │
├────┼─────────────────────┼────────────────────────┼───────────┼────────────┼───────────────┤
│  1 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 62.780 GiB │    64.000 KiB │
│  2 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 62.780 GiB │    64.000 KiB │
│  3 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 62.780 GiB │    64.000 KiB │
│  4 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 62.780 GiB │    64.000 KiB │
└────┴─────────────────────┴────────────────────────┴───────────┴────────────┴───────────────┘

Reproducing the bug

1. Describe what's not working.

AMDGPU.jl tests do not pass, and after a quick inspection AMDGPU can't allocate simple arrays on MI300A. We aim to add MI300A to JACC's CI, so there will be some form of testing after this issue is resolved.

2. Provide MWE to reproduce it (if possible).

The following also does not work for array allocations in general.

julia> using AMDGPU

julia> AMDGPU.zeros(Float32, 10)
ERROR: Failed to successfully execute function and free resources for it.
Reporting current memory usage:
- HIP pool used: 0 bytes.
- HIP pool reserved: 0 bytes.
- Hard memory limit: none.

Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:44
  [2] alloc_or_retry!(f::AMDGPU.Runtime.Mem.var"#5#6"{…}, isfailed::typeof(isnothing); stream::HIPStream)
    @ AMDGPU.Runtime.Mem ~/codes/AMDGPU.jl/src/runtime/memory/utils.jl:38
  [3] AMDGPU.Runtime.Mem.HIPBuffer(bytesize::Int64; stream::HIPStream)
    @ AMDGPU.Runtime.Mem ~/codes/AMDGPU.jl/src/runtime/memory/hip.jl:45
  [4] HIPBuffer
    @ ~/codes/AMDGPU.jl/src/runtime/memory/hip.jl:38 [inlined]
  [5] macro expansion
    @ ~/codes/AMDGPU.jl/src/memory.jl:476 [inlined]
  [6] macro expansion
    @ ./timing.jl:461 [inlined]
  [7] pool_alloc(::Type{AMDGPU.Runtime.Mem.HIPBuffer}, bytesize::Int64)
    @ AMDGPU ~/codes/AMDGPU.jl/src/memory.jl:474
  [8] (::AMDGPU.var"#29#30"{AMDGPU.Runtime.Mem.HIPBuffer, Float32, Tuple{Int64}, Int64})()
    @ AMDGPU ~/codes/AMDGPU.jl/src/array.jl:11
  [9] cached_alloc(f::AMDGPU.var"#29#30"{…}, key::Tuple{…})
    @ GPUArrays ~/.julia-1.12-faraday/packages/GPUArrays/VNJ6B/src/host/alloc_cache.jl:36
 [10] ROCArray{Float32, 1, AMDGPU.Runtime.Mem.HIPBuffer}(::UndefInitializer, dims::Tuple{Int64})
    @ AMDGPU ~/codes/AMDGPU.jl/src/array.jl:9
 [11] ROCArray
    @ ~/codes/AMDGPU.jl/src/array.jl:138 [inlined]
 [12] (ROCArray{Float32})(::UndefInitializer, dims::Int64)
    @ AMDGPU ~/codes/AMDGPU.jl/src/array.jl:156
 [13] zeros(T::Type, dims::Int64)
    @ AMDGPU ~/codes/AMDGPU.jl/src/array.jl:308
 [14] top-level scope
    @ REPL[4]:1
Some type information was truncated. Use `show(err)` to see complete types.

The same AMDGPU.jl code works on MI100 and MI250X

[luraess]

Thanks for reporting. This is weird though as AMDGPU CI is since recently also running on MI300a and all tests are passing there, see e.g. https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/3266148763123078/6677263153381401/-/jobs/14970146912

Can you make sure you are on latest AMDGPU and possibly post the output of the Pkg.status()?

[williamfgc]

@luraess thanks! I am testing the latest AMDGPU.jl see the output of status:

(AMDGPU) pkg> st
Project AMDGPU v2.5.1
Status `~/codes/AMDGPU.jl/Project.toml`
  [621f4979] AbstractFFTs v1.5.0
  [6a4ca0a5] AcceleratedKernels v0.4.3
  [79e6a3ab] Adapt v4.7.0
  [a9b6321e] Atomix v1.1.3
  [ab4f0b2a] BFloat16s v0.6.1
  [fa961155] CEnum v0.5.0
  [e2ba6199] ExprTools v0.1.10
  [0c68f7d7] GPUArrays v11.5.8
  [61eb1bfa] GPUCompiler v1.22.7
⌅ [096a3bc2] GPUToolbox v1.1.1
  [63c18a36] KernelAbstractions v0.9.41
  [929cbde3] LLVM v9.10.0
  [21216c6a] Preferences v1.5.2
  [08abe8d2] PrettyTables v3.3.2
  [74087812] Random123 v1.7.1
  [e6cf234a] RandomNumbers v1.6.0
  [276daf66] SpecialFunctions v2.8.0
  [1e83bf80] StaticArraysCore v1.4.4
  [10745b16] Statistics v1.11.1
  [013be700] UnsafeAtomics v0.3.1
⌅ [86de99a1] LLVM_jll v18.1.7+4
⌅ [873c0968] ROCmDeviceLibs_jll v6.2.1+2
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.12.0
  [44cfe95a] Pkg v1.12.1
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [2f01184e] SparseArrays v1.12.0
  [d55e3150] LLD_jll v18.1.7+5
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated`

[luraess]

Thanks. I just tested interactively again on the CI supercomputer running MI300A and both using AMDGPU v2.5.1 as well as #main seems to work without issue there:

[beverin][lraess@nid002710 ~]$ julia12
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.6 (2026-04-09)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org release
|__/                   |

(@v1.12) pkg> activate --temp
  Activating new project at `/tmp/jl_hium6Q`

(jl_hium6Q) pkg> add AMDGPU
    Updating registry at `~/julia_amd/registries/General.toml`
   Resolving package versions...
   Installed Adapt ─────── v4.7.0
   Installed GPUCompiler ─ v1.22.7
   Installed LLVM ──────── v9.10.0
    Updating `/tmp/jl_hium6Q/Project.toml`
  [21141c5a] + AMDGPU v2.5.1
    Updating `/tmp/jl_hium6Q/Manifest.toml`
  [21141c5a] + AMDGPU v2.5.1
  [621f4979] + AbstractFFTs v1.5.0
  [6a4ca0a5] + AcceleratedKernels v0.4.3
  [79e6a3ab] + Adapt v4.7.0
  [dce04be8] + ArgCheck v2.5.0
  [a9b6321e] + Atomix v1.1.3
  [ab4f0b2a] + BFloat16s v0.6.1
  [fa961155] + CEnum v0.5.0
  [a8cc5b0e] + Crayons v4.1.1
  [9a962f9c] + DataAPI v1.16.0
  [e2d170a0] + DataValueInterfaces v1.0.0
  [ffbed154] + DocStringExtensions v0.9.5
  [e2ba6199] + ExprTools v0.1.10
  [0c68f7d7] + GPUArrays v11.5.8
  [46192b85] + GPUArraysCore v0.2.0
  [61eb1bfa] + GPUCompiler v1.22.7
⌅ [096a3bc2] + GPUToolbox v1.1.1
  [076d061b] + HashArrayMappedTries v0.2.0
  [92d709cd] + IrrationalConstants v0.2.6
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.8.0
  [63c18a36] + KernelAbstractions v0.9.41
  [929cbde3] + LLVM v9.10.0
  [b964fa9f] + LaTeXStrings v1.4.0
  [2ab3a3ac] + LogExpFunctions v1.0.1
  [1914dd2f] + MacroTools v0.5.16
⌅ [bac558e1] + OrderedCollections v1.8.2
  [aea7be01] + PrecompileTools v1.3.4
  [21216c6a] + Preferences v1.5.2
  [08abe8d2] + PrettyTables v3.3.2
  [74087812] + Random123 v1.7.1
  [e6cf234a] + RandomNumbers v1.6.0
  [189a3867] + Reexport v1.2.2
  [ae029012] + Requires v1.3.1
  [7e506255] + ScopedValues v1.6.2
  [6c6a2e73] + Scratch v1.3.0
  [276daf66] + SpecialFunctions v2.8.0
  [90137ffa] + StaticArrays v1.9.18
  [1e83bf80] + StaticArraysCore v1.4.4
  [10745b16] + Statistics v1.11.1
  [892a3eda] + StringManipulation v0.4.4
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.12.1
  [e689c965] + Tracy v0.1.6
  [013be700] + UnsafeAtomics v0.3.1
  [dad2f222] + LLVMExtra_jll v0.0.43+1
⌅ [86de99a1] + LLVM_jll v18.1.7+4
  [ad6e5548] + LibTracyClient_jll v0.13.1+0
  [efe28fd5] + OpenSpecFun_jll v0.5.6+0
⌅ [873c0968] + ROCmDeviceLibs_jll v6.2.1+2
  [0dad84c5] + ArgTools v1.1.2
  [56f22d72] + Artifacts v1.11.0
  [2a0f44e3] + Base64 v1.11.0
  [ade2ca70] + Dates v1.11.0
  [f43a241f] + Downloads v1.7.0
  [7b1f6079] + FileWatching v1.11.0
  [b77e0a4c] + InteractiveUtils v1.11.0
  [ac6e5ff7] + JuliaSyntaxHighlighting v1.12.0
  [4af54fe1] + LazyArtifacts v1.11.0
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2 v1.11.0
  [8f399da3] + Libdl v1.11.0
  [37e2e46d] + LinearAlgebra v1.12.0
  [56ddb016] + Logging v1.11.0
  [d6f4376e] + Markdown v1.11.0
  [ca575930] + NetworkOptions v1.3.0
  [44cfe95a] + Pkg v1.12.1
  [de0858da] + Printf v1.11.0
  [3fa0cd96] + REPL v1.11.0
  [9a3f8284] + Random v1.11.0
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization v1.11.0
  [6462fe0b] + Sockets v1.11.0
  [2f01184e] + SparseArrays v1.12.0
  [f489334b] + StyledStrings v1.11.0
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [cf7118a7] + UUIDs v1.11.0
  [4ec0a83e] + Unicode v1.11.0
  [e66e0078] + CompilerSupportLibraries_jll v1.3.0+1
  [d55e3150] + LLD_jll v18.1.7+5
  [deac9b47] + LibCURL_jll v8.15.0+0
  [e37daf67] + LibGit2_jll v1.9.0+0
  [29816b5a] + LibSSH2_jll v1.11.3+1
  [14a3606d] + MozillaCACerts_jll v2025.11.4
  [4536629a] + OpenBLAS_jll v0.3.29+0
  [05823500] + OpenLibm_jll v0.8.7+0
  [458c3c95] + OpenSSL_jll v3.5.4+0
  [bea87d4a] + SuiteSparse_jll v7.8.3+2
  [83775a58] + Zlib_jll v1.3.1+2
  [8f36deef] + libLLVM_jll v18.1.7+5
  [8e850b90] + libblastrampoline_jll v5.15.0+0
  [8e850ede] + nghttp2_jll v1.64.0+1
  [3f19e933] + p7zip_jll v17.7.0+0
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling packages finished.
  15 dependencies successfully precompiled in 53 seconds. 79 already precompiled.

(jl_hium6Q) pkg> st
Status `/tmp/jl_hium6Q/Project.toml`
  [21141c5a] AMDGPU v2.5.1

julia> using AMDGPU
┌ Warning: MIOpen is unavailable, functionality will be disabled.
└ @ AMDGPU ~/julia_amd/packages/AMDGPU/Z5AP0/src/AMDGPU.jl:213

julia> AMDGPU.versioninfo()
[ Info: AMDGPU versioninfo
┌───────────┬──────────────────┬───────────┬───────────────────────────────────────────────────────────────────────────────────
│ Available │ Name             │ Version   │ Path                                                                             ⋯
├───────────┼──────────────────┼───────────┼───────────────────────────────────────────────────────────────────────────────────
│     +     │ LLD              │ -         │ /user-environment/env/default/bin/ld.lld                                         ⋯
│     +     │ Device Libraries │ -         │ /users/lraess/julia_amd/artifacts/0a0cad46b7dacc66387ed19e8c0f45b097f46691/amdgc ⋯
│     +     │ HIP              │ 7.2.26103 │ /user-environment/env/default/lib/libamdhip64.so                                 ⋯
│     +     │ rocBLAS          │ 5.2.0     │ /user-environment/env/default/lib/librocblas.so                                  ⋯
│     +     │ rocSOLVER        │ 3.32.0    │ /user-environment/env/default/lib/librocsolver.so                                ⋯
│     +     │ rocSPARSE        │ 4.2.0     │ /user-environment/env/default/lib/librocsparse.so                                ⋯
│     +     │ rocRAND          │ 4.2.0     │ /user-environment/env/default/lib/librocrand.so                                  ⋯
│     +     │ rocFFT           │ 1.0.36    │ /user-environment/env/default/lib/librocfft.so                                   ⋯
│     -     │ MIOpen           │ -         │ -                                                                                ⋯
└───────────┴──────────────────┴───────────┴───────────────────────────────────────────────────────────────────────────────────
                                                                                                               1 column omitted

[ Info: AMDGPU devices
┌────┬─────────────────────┬────────────────────────┬───────────┬─────────────┬───────────────┐
│ Id │                Name │               GCN arch │ Wavefront │      Memory │ Shared Memory │
├────┼─────────────────────┼────────────────────────┼───────────┼─────────────┼───────────────┤
│  1 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 128.000 GiB │    64.000 KiB │
│  2 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 128.000 GiB │    64.000 KiB │
│  3 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 128.000 GiB │    64.000 KiB │
│  4 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 128.000 GiB │    64.000 KiB │
└────┴─────────────────────┴────────────────────────┴───────────┴─────────────┴───────────────┘


julia> AMDGPU.zeros(Float32, 10)
10-element ROCArray{Float32, 1, AMDGPU.Runtime.Mem.HIPBuffer}:
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0

julia> 
[beverin][lraess@nid002710 lraess]$ cd AMDGPU.jl/
[beverin][lraess@nid002710 AMDGPU.jl]$ git status
On branch main
Your branch is up to date with 'origin/main'.

[beverin][lraess@nid002710 AMDGPU.jl]$ julia12 --project
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.6 (2026-04-09)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org release
|__/                   |

(AMDGPU) pkg> st
Project AMDGPU v2.5.1
Status `/capstor/scratch/cscs/lraess/AMDGPU.jl/Project.toml`
  [621f4979] AbstractFFTs v1.5.0
  [6a4ca0a5] AcceleratedKernels v0.4.3
  [79e6a3ab] Adapt v4.7.0
  [a9b6321e] Atomix v1.1.3
  [ab4f0b2a] BFloat16s v0.6.1
  [fa961155] CEnum v0.5.0
  [e2ba6199] ExprTools v0.1.10
  [0c68f7d7] GPUArrays v11.5.8
  [61eb1bfa] GPUCompiler v1.22.7
⌅ [096a3bc2] GPUToolbox v1.1.1
  [63c18a36] KernelAbstractions v0.9.41
  [929cbde3] LLVM v9.10.0
  [21216c6a] Preferences v1.5.2
  [08abe8d2] PrettyTables v3.3.2
  [74087812] Random123 v1.7.1
  [e6cf234a] RandomNumbers v1.6.0
  [276daf66] SpecialFunctions v2.8.0
  [1e83bf80] StaticArraysCore v1.4.4
  [10745b16] Statistics v1.11.1
  [013be700] UnsafeAtomics v0.3.1
⌅ [86de99a1] LLVM_jll v18.1.7+4
⌅ [873c0968] ROCmDeviceLibs_jll v6.2.1+2
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.12.0
  [44cfe95a] Pkg v1.12.1
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [2f01184e] SparseArrays v1.12.0
  [d55e3150] LLD_jll v18.1.7+5
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated`

julia> using AMDGPU
┌ Warning: MIOpen is unavailable, functionality will be disabled.
└ @ AMDGPU /capstor/scratch/cscs/lraess/AMDGPU.jl/src/AMDGPU.jl:213

julia> AMDGPU.versioninfo()
AMDGPU versioninfo
┌───────────┬──────────────────┬───────────┬───────────────────────────────────────────────────────────────────────────────────
│ Available │ Name             │ Version   │ Path                                                                             ⋯
├───────────┼──────────────────┼───────────┼───────────────────────────────────────────────────────────────────────────────────
│     +     │ LLD              │ -         │ /user-environment/env/default/bin/ld.lld                                         ⋯
│     +     │ Device Libraries │ -         │ /users/lraess/julia_amd/artifacts/0a0cad46b7dacc66387ed19e8c0f45b097f46691/amdgc ⋯
│     +     │ HIP              │ 7.2.26103 │ /user-environment/env/default/lib/libamdhip64.so                                 ⋯
│     +     │ rocBLAS          │ 5.2.0     │ /user-environment/env/default/lib/librocblas.so                                  ⋯
│     +     │ rocSOLVER        │ 3.32.0    │ /user-environment/env/default/lib/librocsolver.so                                ⋯
│     +     │ rocSPARSE        │ 4.2.0     │ /user-environment/env/default/lib/librocsparse.so                                ⋯
│     +     │ rocRAND          │ 4.2.0     │ /user-environment/env/default/lib/librocrand.so                                  ⋯
│     +     │ rocFFT           │ 1.0.36    │ /user-environment/env/default/lib/librocfft.so                                   ⋯
│     -     │ MIOpen           │ -         │ -                                                                                ⋯
└───────────┴──────────────────┴───────────┴───────────────────────────────────────────────────────────────────────────────────
                                                                                                               1 column omitted

AMDGPU devices
┌────┬─────────────────────┬────────────────────────┬───────────┬─────────────┬───────────────┐
│ Id │                Name │               GCN arch │ Wavefront │      Memory │ Shared Memory │
├────┼─────────────────────┼────────────────────────┼───────────┼─────────────┼───────────────┤
│  1 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 128.000 GiB │    64.000 KiB │
│  2 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 128.000 GiB │    64.000 KiB │
│  3 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 128.000 GiB │    64.000 KiB │
│  4 │ AMD Instinct MI300A │ gfx942:sramecc+:xnack- │        64 │ 128.000 GiB │    64.000 KiB │
└────┴─────────────────────┴────────────────────────┴───────────┴─────────────┴───────────────┘


julia> AMDGPU.zeros(Float32, 10)
10-element ROCArray{Float32, 1, AMDGPU.Runtime.Mem.HIPBuffer}:
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0

julia> 

This may suggest a possible issue on your side outside of AMDGPU. How was ROCm installed there? is it an Ubuntu machine? This may be a possible pointer #901.

[williamfgc]

@luraess thanks a lot for the pointer and background information. Indeed I launched your reproducer in #901 and it signals the same issue:

./test-hipMalloc 
Device: AMD Instinct MI300A  arch: gfx942:sramecc+:xnack-

[hipMalloc]             OK
[hipMallocAsync]        out of memory
[hipMallocFromPoolAsync] out of memory

This would make any Julia >= 12 + recent AMDGPU.jl combination fail.
I'll check with our sys admins and follow your suggested fix form the ROCm side.

[luraess]

Thanks for the feedback. It's not a Julia issue, but it could be that drivers install may not be optimal. We've hit this issue on a Ubuntu 24 LTS system where drivers for ROCm 7.x were not installed following the latest procedure (see #901 (comment)).

[williamfgc]

@luraess I can confirm this was indeed the issue. Our system admins fixed the drivers issue and now it works. You're a rockstar, thanks!

[luraess]

Glad you could fix it!