Volatile markets in which stocks move less in lockstep should be a recipe for making money. But much of the industry is struggling, and clients are losing patience.