failed due to "***Memory realloc failed for cnbrpoolGet: cnbrpool. Requested size: 1696128 bytes"

I have a very large FEM model ( No. of element: 19,294,342, No. of nodes: 3,403,564 ). I use parMetis to partiton the model(ParMETIS_V3_PartMeshKway). The code works very well when I use 512 processors. But it failed when I attempt to use 640 processors (same error when I use 1024 processors). The code did not return the partition. And the error is ( Maximum memory used: 53727992 bytes, Current memory used: 30653200 bytes, ***Memory realloc failed for cnbrpoolGet: cnbrpool. Requested size: 1696128 bytes ). And I located the function (idx_t cnbrpoolGetNext(ctrl_t *ctrl, idx_t nnbrs)).

I am wondering what causes the error, is it my own fem code, or the parMetis, or else?

Thanks in advance.