There is a big space of possible solutions, but in general any solution where you lose research-production parity that dramatically will make it very hard to play around with the production environment.
This is one way to do it, and probably the preferred way if you are a solo trader or a small team. In a big team with separate research/dev then you probably want research to generate the features, devs to implement them for prod, and have a *very* careful reconciliation.https://twitter.com/cryptoquantHQ/status/1458476996900556807 …
One mistake I used to always make was having totally separate ML and production pipelines/infra. I used to do all of my feature engineering in Jupyter notebooks using handy pandas.rolling() methods and easy visualization.
In my opinion, this is the wrong approach.
1/5 
Show this thread
-
-
-
A lot of quant research involves small tweaks to a large and complex system, and as soon as researchers cannot do that themselves, eg, make a small change to a single feature and run a sim, then you pay a large productivity cost.
End of conversation
New conversation -
-
-
Makes perfect sense for separate research/dev pipelines in large teams as they require different skill sets and likely different languages (python/c++). As a solo trader though, I was generating a lot of extra work, particularly with reconciliation.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Yup. Multiple correct approaches on this depending on size, frequency, etc.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
I think generally modular build mindset is a safe path
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
I don’t see the advantage to doing this. One thing you can do is have your strategy/research code in Python with the underlying classes in a compiled language. Then the Python stuff is essentially configuration and everything that gets run is in a fast language. 1/N
-
You then have basic building blocks for researchers like math, logic, and signal processing classes plus stuff like state machines. If a researcher finds something good then a dev or said researcher can write a class specific to that signal which is much faster.
- Show replies
New conversation -
-
-
How tight are the tolerances? IMO ML is a great tool for a “close enough” answer. Not saying I’d accept waste, but I’m more likely to squeeze PnL out of a notebook I understand than a black box script.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.