gmx::PmePpCommGpu Class Reference

#include <gromacs/ewald/pme_pp_comm_gpu.h>

Description

Manages communication related to GPU buffers between this PME rank and its PP rank.

Classes
class	Impl
	Class with interfaces and data for GPU version of PME-PP Communication. More...

Public Member Functions
	PmePpCommGpu (MPI_Comm comm, int pmeRank, gmx::HostVector< gmx::RVec > *pmeCpuForceBuffer, const DeviceContext &deviceContext, const DeviceStream &deviceStream, bool useNvshmem)
	Creates PME-PP GPU communication object. More...
void	reinit (int size)
	Perform steps required when buffer size changes. More...
void	receiveForceFromPme (RVec *recvPtr, int recvSize, bool recvPmeForceToGpu)
	Pull data from PME GPU directly using CUDA Memory copy. More...
void	sendCoordinatesToPmeFromGpu (DeviceBuffer< RVec > sendPtr, int sendSize, GpuEventSynchronizer *coordinatesReadyOnDeviceEvent, bool receiveForcesToGpu)
	Push coordinates buffer directly to GPU memory on PME task Note that, when using GPU-aware MPI with staged communication and not using NVSHMEM for GPU force receives, this method also posts a non-blocking force receive request to overlap communication with computation. More...
void	sendCoordinatesToPmeFromCpu (const RVec *sendPtr, int sendSize, bool receiveForcesToGpu)
	Push coordinates buffer from host memory directly to GPU memory on PME task. More...
std::optional< DeviceBuffer < RVec > >	getGpuForceStagingPtr ()
	When this PP rank has particles with PME force contributions expected from its PME-only rank, return pointer to buffer used for staging PME force on GPU.
std::optional < GpuEventSynchronizer * >	getForcesReadySynchronizer ()
	When this thread-MPI rank has particles with PME force contribtions expected from its PME-only rank, return pointer to event recorded when forces are ready.
DeviceBuffer< uint64_t >	getGpuForcesSyncObj ()
	Return pointer to force synchronization NVSHMEM object.

Constructor & Destructor Documentation

gmx::PmePpCommGpu::PmePpCommGpu	(	MPI_Comm	comm,
		int	pmeRank,
		gmx::HostVector< gmx::RVec > *	pmeCpuForceBuffer,
		const DeviceContext &	deviceContext,
		const DeviceStream &	deviceStream,
		bool	useNvshmem
	)

Creates PME-PP GPU communication object.

Constructor stub.

Parameters

[in]	comm	Communicator used for simulation
[in]	pmeRank	Rank of PME task
[in]	pmeCpuForceBuffer	Buffer for PME force in CPU memory
[in]	deviceContext	GPU context.
[in]	deviceStream	GPU stream.
[in]	useNvshmem	NVSHMEM enable/disable for GPU comm.

Member Function Documentation

void gmx::PmePpCommGpu::receiveForceFromPme	(	RVec *	recvPtr,
		int	recvSize,
		bool	recvPmeForceToGpu
	)

Pull data from PME GPU directly using CUDA Memory copy.

Parameters

[out]	recvPtr	Buffer to receive PME force data
[in]	recvSize	Number of elements to receive
[in]	recvPmeForceToGpu	Whether receive is to GPU, otherwise CPU

void gmx::PmePpCommGpu::reinit

(

int

size

)

Perform steps required when buffer size changes.

init PME-PP GPU communication stub

Parameters

[in]

size

Number of elements in buffer

void gmx::PmePpCommGpu::sendCoordinatesToPmeFromCpu	(	const RVec *	sendPtr,
		int	sendSize,
		bool	receiveForcesToGpu
	)

Push coordinates buffer from host memory directly to GPU memory on PME task.

Parameters

[in]	sendPtr	Buffer with coordinate data
[in]	sendSize	Number of elements to send
[in]	receiveForcesToGpu	Whether PME forces will be received to GPU

void gmx::PmePpCommGpu::sendCoordinatesToPmeFromGpu	(	DeviceBuffer< RVec >	sendPtr,
		int	sendSize,
		GpuEventSynchronizer *	coordinatesReadyOnDeviceEvent,
		bool	receiveForcesToGpu
	)

Push coordinates buffer directly to GPU memory on PME task Note that, when using GPU-aware MPI with staged communication and not using NVSHMEM for GPU force receives, this method also posts a non-blocking force receive request to overlap communication with computation.

Parameters

[in]	sendPtr	Buffer with coordinate data
[in]	sendSize	Number of elements to send
[in]	coordinatesReadyOnDeviceEvent	Event recorded when coords available on device
[in]	receiveForcesToGpu	Whether PME forces will be received to GPU

---

The documentation for this class was generated from the following files:

• src/gromacs/ewald/pme_pp_comm_gpu.h
• src/gromacs/ewald/pme_pp_comm_gpu_impl.cpp
• src/gromacs/ewald/pme_pp_comm_gpu_impl_gpu.cpp