New roundup issue: Data Mesh, Contracts, and Distributed Ownership.https://roundup.getdbt.com/p/data-mesh-contracts-and-distributed?r=23p7n&utm_campaign=post&utm_medium=web&utm_source=twitter …
-
But all the problems you cite can manifest themselves in code for analysis too! Data people love to bemoan how their analysis patterns are too unique or custom for standardization, and while I'm sympathetic, I'm not convinced it's impossible
In many cases, the assumptions of an analysis framework or statistical model are function like the integration points or APIs you mention, and some of the biggest problems arise from practitioners not understanding or willfully ignoring those assumptions
-
-
Just look at A/B tests. They are an incredibly widespread form of standardized data analysis, and one of the things that is most likely to make them worthless interpretation-wise is a violation of the stable unit treatment value assumption
-
I like the idea of the data mesh, but it's only one piece of the puzzle. Data warehouses being a type of data product that connect to other data products (like experimentation platforms or funnel analysis tools) through standardized APIs is IMO a helpful extension of the concept
End of conversation
New conversation -
-
-
Another related. You're on point!https://twitter.com/statwonk/status/1428839542430306310 …
Given data warehouses are designed for dashboards / BI, data leakage is one of the largest hurdles facing machine learning and data science teams today (endogeneity / simultaneity). It’ll be cool to see if it’s something feature stores improve.
Show this threadThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.