Take the skillet off the burner and mix in the potatoes and olives. Incorporate the diced hard-boiled eggs and allow the mixture to sit and cool for roughly 20 minutes.