ULTRARAM is an emerging memory technology exhibiting high endurance (> 10^7 P/E cycles), ultra-high retention (>1000 years), and ultra-low switching energy per unit area. This compound semiconductor-based non-volatile memory (NVM) works on the principle of triple-barrier resonant tunneling (TBRT) using InAs/AlSb heterostructures. Single memory cells are fabricated on GaAs and Si substrates. In addition, a physics-based compact model has been proposed to accurately capture the real-time trapping/de-trapping of charges in the floating gate (FG) and utilized for the synapse simulations. Array-level simulations and benchmarking highlight the promise of this technology as a next-generation non-volatile memory cells. Additionally, a circuit-level macro-model is employed to evaluate and benchmark the on-chip learning performance in terms of area, latency, energy, and accuracy of an ULTRARAM synaptic core.