Private equity firm Genstar Capital said on Thursday it will buy asset management software company Prometheus Group from private equity firm Francisco Partners for over $1 billion.