this is actually deeply irritating to me ive worked at lots of tech companies large and small in data/ml roles you have to understand every one of them is held together with the equivalent of duct tape and fortuitously growing barnacles, no one understands how the systems workhttps://twitter.com/eigenrobot/status/1106374706674847745 …
-
Show this thread
-
in 2015 or so I did some data pipelining at Amazon, the premier cloud data place most of my time was spent building and refining a script that let me work around problems inherent to their *internal* ETL platform go and ask an AMZN veteran about data warehouse they might cry
2 replies 1 retweet 30 likesShow this thread -
every product team has their own algorithms and often the use of these algorithms is to undermine algorithms operating on another part of the site over which a team has no control most of these algorithms were written by people who left the company and left no documentation
3 replies 2 retweets 35 likesShow this thread -
another team I worked on had a production pipeline that consisted of a bunch of R scripts with hard-coded dates that had to be manually updated every month and run in a particular order while manually moving files around this was at a company youve heard of
3 replies 0 retweets 32 likesShow this thread -
and this is all just data work I have no idea what the poor devs and ML engineers are working with but bless them
4 replies 0 retweets 19 likesShow this thread -
Replying to @eigenrobot
As if THE GOVERNMENT ITSELF isn’t held together by “duct tape and barnacles”
1 reply 0 retweets 3 likes
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.