Issue: While running DCDIAG you receive the following error: “The replication operation failed to allocate memory”.
Resolution: I checked the page file settings as well as reviewed Task Manager to confirm the server was not out of memory. I decided to reboot the server to see if the memory issue would go away and in my case the issue was resolved. This particular domain controller was Virtualized with VMWare so I granted it a small memory reservation (256 MB) in VMWare.
Additional Circumstances: I’m still investigating this issue, in this case I saw that AD replication had stopped, numerous Time Sync issues were found as well as serious DNS errors to the affect that DNS Server could not find a DNS. I’m not sure what issue precipitated the others, my guess is the memory issue lead to the cease of AD replication and DNS issue and the Time Sync issue was a separate issue.