There are plenty of differences between DS and SWE, but one similarity that DS tend to miss is that the one-off-task-that-you-swear-is-just-a-one-off rarely is something you actually only do once.
-
Show this thread
-
DS tend to be more conscious of this when it comes to analysis, hence the emphasis on reproducibility in some circles. But analysis is not the only thing DS teams produce, and it's probably not the likeliest to cause unforeseen pain down the road.
1 reply 0 retweets 5 likesShow this thread -
Anecdotally, I have observed that things DS don't like doing or don't think of as their job, are particularly prone to becoming the one-off-tasks-you-swear-are-just-one-offs that come back to bite you
1 reply 0 retweets 4 likesShow this thread -
Katie Bauer! Retweeted Will Geary
Dashboards are the poster child of this phenomenon, and many bundles of 280 characters have been dedicated to vilifying them.https://mobile.twitter.com/wgeary/status/1379575455448186883 …
Katie Bauer! added,
1 reply 0 retweets 6 likesShow this thread -
The perception of dashboards as a waste of time is interesting to me, perhaps because I was a business analyst early in my data-world-life. I was primarily responsible for pulling reports and spent a lot of time trying to script them so I could do more "promotion-worthy" work
1 reply 0 retweets 4 likesShow this thread -
The scripting was kinda fun, but I'd be lying if I didn't say using a proper BI tool felt like a weight lifted off my shoulders. I had some high ranking people expecting my reports, and when they didn't receive them because my script failed overnight, I DEFINITELY heard about it
1 reply 0 retweets 3 likesShow this thread -
BI tools and dashboards enabled me to build reports scalably because they removed a ton of maintenance overhead, and it made it much easier to create more reports than I ever previously could. But that exact same lack of friction meant I got a little careless.
1 reply 0 retweets 6 likesShow this thread -
They were easy to use, so I used them for everything. They were my analysis container, a UI for "tools" that I'd created, the way I shared a bunch of related code snippets... and while these tools were created for automating reports, they were not built for those other purposes.
1 reply 0 retweets 5 likesShow this thread -
And while I had a sense of what it meant to maintain a report, I didn't really consider that having those other things available in the same format implied that they'd have the same level of support. I was accidentally building infrastructure. Rickety, rickety infrastructure.
1 reply 0 retweets 7 likesShow this thread
Not being a SWE, it seems to me like this happens less often in their world (although maybe not, given how buried everyone seems to be in tech debt). I think this must partly be because engineering focuses on reliability by default, whereas science does not.
-
-
But IMO it ultimately comes back to problem of organizations not knowing what they want from Data Science, and it requires DS to be clear about the longevity of what it produces. Not everything should be infrastructure, but stakeholders may not realize unless it's made clear
2 replies 0 retweets 18 likesShow this threadThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.

