As quants adapt their models to a Chinese market where mom-and-pop investors drive more than 80% of trades, they’re coming up with clever ways to predict where the nation’s so-called dumb money is headed.