There are plenty of differences between DS and SWE, but one similarity that DS tend to miss is that the one-off-task-that-you-swear-is-just-a-one-off rarely is something you actually only do once.
Show this thread
-
The scripting was kinda fun, but I'd be lying if I didn't say using a proper BI tool felt like a weight lifted off my shoulders. I had some high ranking people expecting my reports, and when they didn't receive them because my script failed overnight, I DEFINITELY heard about it
Show this thread
BI tools and dashboards enabled me to build reports scalably because they removed a ton of maintenance overhead, and it made it much easier to create more reports than I ever previously could. But that exact same lack of friction meant I got a little careless.
-
-
They were easy to use, so I used them for everything. They were my analysis container, a UI for "tools" that I'd created, the way I shared a bunch of related code snippets... and while these tools were created for automating reports, they were not built for those other purposes.
Show this thread -
And while I had a sense of what it meant to maintain a report, I didn't really consider that having those other things available in the same format implied that they'd have the same level of support. I was accidentally building infrastructure. Rickety, rickety infrastructure.
Show this thread -
Not being a SWE, it seems to me like this happens less often in their world (although maybe not, given how buried everyone seems to be in tech debt). I think this must partly be because engineering focuses on reliability by default, whereas science does not.
Show this thread -
But IMO it ultimately comes back to problem of organizations not knowing what they want from Data Science, and it requires DS to be clear about the longevity of what it produces. Not everything should be infrastructure, but stakeholders may not realize unless it's made clear
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.