gmx::PmePpCommGpu Class Reference

#include <gromacs/ewald/pme_pp_comm_gpu.h>

Description

Manages communication related to GPU buffers between this PP rank and its PME rank.

Note that the coordinates and forces are transferred between each on each step, even from empty domains. However the long-range forces only participate in the subsequent reduction phase when the domain has particles.

Public Member Functions
	PmePpCommGpu (MPI_Comm comm, int pmeRank, gmx::HostVector< gmx::RVec > *pmeCpuForceBuffer, const DeviceContext &deviceContext, const DeviceStream &deviceStream, bool useNvshmem)
	Creates PME-PP GPU communication object. More...
void	reinit (int size)
	Perform steps required when buffer size changes. More...
void	receiveForceFromPme (RVec *recvPtr, int recvSize, bool recvPmeForceToGpu)
	Pull data from PME GPU directly using GPU Memory copy. More...
void	sendCoordinatesToPmeFromGpu (DeviceBuffer< RVec > sendPtr, int sendSize, GpuEventSynchronizer *coordinatesReadyOnDeviceEvent, bool receiveForcesToGpu)
	Push coordinates buffer directly to GPU memory on PME task Note that, when using GPU-aware MPI with staged communication and not using NVSHMEM for GPU force receives, this method also posts a non-blocking force receive request to overlap communication with computation. More...
void	sendCoordinatesToPmeFromCpu (const RVec *sendPtr, int sendSize, bool receiveForcesToGpu)
	Push coordinates buffer from host memory directly to GPU memory on PME task. More...
std::optional< DeviceBuffer < RVec > >	getGpuForceStagingPtr ()
	When this PP rank has particles with PME force contributions expected from its PME-only rank, return pointer to buffer used for staging PME force on GPU.
std::optional < GpuEventSynchronizer * >	getForcesReadySynchronizer ()
	When this thread-MPI rank has particles with PME force contribtions expected from its PME-only rank, return pointer to event recorded when forces are ready.
DeviceBuffer< uint64_t >	getGpuForcesSyncObj ()
	Return pointer to force synchronization NVSHMEM object.

Constructor & Destructor Documentation

gmx::PmePpCommGpu::PmePpCommGpu	(	MPI_Comm	comm,
		int	pmeRank,
		gmx::HostVector< gmx::RVec > *	pmeCpuForceBuffer,
		const DeviceContext &	deviceContext,
		const DeviceStream &	deviceStream,
		bool	useNvshmem
	)

Creates PME-PP GPU communication object.

Constructor stub.

Parameters

[in]	comm	Communicator used for simulation
[in]	pmeRank	Rank of PME task
[in]	pmeCpuForceBuffer	Buffer for PME force in CPU memory
[in]	deviceContext	GPU context.
[in]	deviceStream	GPU stream.
[in]	useNvshmem	NVSHMEM enable/disable for GPU comm.

Member Function Documentation

void gmx::PmePpCommGpu::receiveForceFromPme	(	RVec *	recvPtr,
		int	recvSize,
		bool	recvPmeForceToGpu
	)

Pull data from PME GPU directly using GPU Memory copy.

Parameters

[out]	recvPtr	Buffer to receive PME force data
[in]	recvSize	Number of elements to receive
[in]	recvPmeForceToGpu	Whether receive is to GPU, otherwise CPU

void gmx::PmePpCommGpu::reinit

(

int

size

)

Perform steps required when buffer size changes.

init PME-PP GPU communication stub

Parameters

[in]

size

Number of elements in buffer

void gmx::PmePpCommGpu::sendCoordinatesToPmeFromCpu	(	const RVec *	sendPtr,
		int	sendSize,
		bool	receiveForcesToGpu
	)

Push coordinates buffer from host memory directly to GPU memory on PME task.

Parameters

[in]	sendPtr	Buffer with coordinate data
[in]	sendSize	Number of elements to send
[in]	receiveForcesToGpu	Whether PME forces will be received to GPU

void gmx::PmePpCommGpu::sendCoordinatesToPmeFromGpu	(	DeviceBuffer< RVec >	sendPtr,
		int	sendSize,
		GpuEventSynchronizer *	coordinatesReadyOnDeviceEvent,
		bool	receiveForcesToGpu
	)

Push coordinates buffer directly to GPU memory on PME task Note that, when using GPU-aware MPI with staged communication and not using NVSHMEM for GPU force receives, this method also posts a non-blocking force receive request to overlap communication with computation.

Parameters

[in]	sendPtr	Buffer with coordinate data
[in]	sendSize	Number of elements to send
[in]	coordinatesReadyOnDeviceEvent	Event recorded when coords available on device
[in]	receiveForcesToGpu	Whether PME forces will be received to GPU

---

The documentation for this class was generated from the following files:

• src/gromacs/ewald/pme_pp_comm_gpu.h
• src/gromacs/ewald/pme_pp_comm_gpu_impl.cpp
• src/gromacs/ewald/pme_pp_comm_gpu_impl_gpu.cpp