Silicon Photonic Accelerated Memory Pooling for Efficient Compute Resource Allocation