Caution: slightly harsh response.
First, kudos! Of the dozens of articles I’ve read on Medium about applying machine learning techniques to investing, however the writer wants to define that term, I’ve never found one to be as well-informed as your post. Thank you. Anybody who intends to write on this topic on Medium in the future is challenged to exceed your post.
And for every writer who has claimed insight or success in some way, none have posted live account P&Ls of any reasonable duration.
Many (most?) people who have data, machine learning expertise, and compute cycles fail to realize investment markets are highly competitive — because there’s a lot of money at stake. Machine learning techniques have been used by financial market participants for decades and there is practically no technique or suite of techniques that haven’t been tested. Many papers have been written, but only on the unsuccessful projects. Nobody writes about the successful ones, until they’re no longer successful — and they’re sitting on a beach with a margarita or traveling the world taking pictures.
Still think you can create the next investing miracle? David did slay Goliath, but keep in mind market-makers compete to buy the order flow from brokers such as Robinhood and others, and have done so for decades. Why? Those market-makers know taking the opposite side of those orders is a nearly guaranteed risk-free gain and are willing to relinquish some of that gain to trade against that order flow.
But, but, Renaissance Technologies! Yes, indeed, RenTech. It closed its Medallion Fund years ago to internal investors, only, at $10B (?), and scales that with leverage to extract all of the signal’s value by varying that leverage. You’ll have to beat it, before you tap out.