JavaScript is not available.

(YC S21)

. Built ML products, now ML tools

. Open source

linkedin.com/in/elenasamuyl…Joined July 2013

327 Following

1,242 Followers

Elena Samuylova’s Tweets

Pinned Tweet

Dec 8, 2022

🚀Excited to announce Evidently 0.2, a stable version of our open-source ML monitoring tool. GitHub: github.com/evidentlyai/ev Blog: evidentlyai.com/blog/evidently Tutorial: docs.evidentlyai.com/tutorial/ 39 releases, 1.5m downloads, and a Big Refactoring later, here what Evidently is 👇

288

Jan 31

“Can I use Evidently to monitor text data?” Yes, you can! With the latest release, you can use Evidently to monitor and evaluate text data quality and drift for your NLP models in production 🔡 Read the release blog 👇 evidentlyai.com/blog/evidently

GIF

Jan 30

Huge kudos to the team for making this release happen! As usual, everything is open-source! GitHub: github.com/evidentlyai/ev Release blog: evidentlyai.com/blog/evidently Text data overview preset (docs): docs.evidentlyai.com/presets/text-o (3/3)

Here is what comes with the new release: 📊A new Text Overview preset. 🔡Domain classifier as a new drift detection method. 🔢Text descriptor drift as an alternative. 📚All relevant existing tests and metrics now support text data. 🗂Multi-modal data support. (2/3)

🚀Super happy to announce the latest

release: text data support! It has been one of the most requested features: “How can I use Evidently to monitor and evaluate text data drift?” Now, you can do that! Our open-source library now supports raw text inputs. (1/3)

Jan 26

Sooner or later, every ML model decays. The world changes, and its static picture (aka train set) becomes obsolete. Often the easiest way to update a model is to retrain it on newer data. Nubank retrains ML models automatically. Here are 5 tips by

@queirozfcom

: 1/6 🧵

Evidently AMA on scaling ML teams and MLOps w/ Neal Lathia · Luma

The folks at

have invited me to an AMA session on their discord on Jan 26. (who has some tips on how to properly use discord?)

lu.ma

Join us for the next text-based AMA session with Neal Lathia to chat about building ML platforms and high-performance ML teams, scaling MLOps, and more. Neal is currently the Staff ML...

Dec 9, 2022

In case you missed it, Evidently 0.2 is here 🚀🚀🚀 39 releases and over 1.5 million downloads later, there are things that our open-source ML monitoring tool does really well 📈 Join our launch event to see what you can do with Evidently 0.2 👇 lu.ma/lqo2oi0i

Evidently Launch Event · Zoom · Luma

Dec 8, 2022

A lot of cool stuff is in the works: 📈More evaluations ⭐️Spark integration 🔡Text data support If you like what we do, support Evidently by giving a ⭐️ on GitHub, and join us to celebrate! GitHub: github.com/evidentlyai/ev Event on Dec 15: lu.ma/lqo2oi0i (4/4)

lu.ma

Join us for the biggest Evidently launch yet! Our Co-founders, Elena Samuylova and Emeli Dral, will spotlight the latest features and the future product vision. Evidently 0.2 is here! We...

📊 A customizable framework for batch model checks. You can mix and match these evaluation “lego bricks” to fit your monitoring scenario. You can also implement your tests and metrics while re-using the underlying Evidently architecture to structure your evaluations. (3/4)

📚An open-source library of ML evaluation metrics and tests. Evidently has over 100 tests and metrics to evaluate data quality, data drift, or ML model performance. Each comes with relevant parameters, clean API, and helpful visualization. Expect more! (2/4)

Anaconda

@anacondainc

Dec 1, 2022

On the latest episode of

@AnacondaInc

’s Numerically Speaking podcast,

@pwang

speaks with

’s

Anaconda | Shaping Best Practices for Monitoring ML Models

about best practices for monitoring #ML models & why it leads to more value. Listen on our website or wherever you get your podcasts 🎙️

anaconda.com

Shaping Best Practices for Monitoring ML Models 7 In this episode, host Peter Wang is joined by Elena Samuylova , CEO and Co-Founder of Evidently AI . Peter and Elena discuss how Evidently AI’s...

Monitoring ML systems in production. Which metrics should you track?

Jul 22, 2022

An ML system has these four components: the software piece, the flowing data, the machine learning model, and the business reason for its existence. Since an ML system is all of the above, the ML monitoring has to be too. The complete blog 👇🏼 evidentlyai.com/blog/ml-monito (6/n)

evidentlyai.com

When one mentions "ML monitoring," this can mean many things. Are you tracking service latency? Model accuracy? Data quality? This blog organizes everything one can look at in a single framework.

🏦 Lastly, we have the business or product KPI. No one uses ML to get "90% accuracy". There is a business need behind it: converting users, getting better forecasts, etc. Does the model bring value to the business? Are the product metrics affected by the model OK? (5/n)

🤖 Third, you have the hero: the ML model itself. No model is perfect, and no model lasts forever. But they still can be useful! You need to ensure it remains so. Is the model still fit for the task? Are the predictions accurate? Can you trust them? (4/n)

📊 Second, you have the data. The production ML models take new data as input. But it changes over time. There are also many issues with data quality and integrity. We must monitor this crucial component. Is the data OK? Can you trust it to generate predictions? (3/n)

💻 First, you still have the software backend. It can be a set of batch jobs or a real-time service. You need to know how this software component works. Did the prediction job execute successfully? Did the service respond? Is it working fast enough? (2/n)

What exactly is ML monitoring? We've published a new entry-level blog on all things it can comprise. When you deploy an ML system in production, it integrates into the business. It's ROI time! To track its performance, you need to look at more than one thing 👇🏼 (1/n)

Timothy Chen

@tnachen

May 5, 2022

The open source ML monitoring companies are on the rise! We sat down with

from

to talk about the journey of building a OSS project and community from scratch at the same time twitter.com/OssStartup/sta

This Tweet is unavailable.

Robby

@amanda_robs

May 5, 2022

This was such a fun convo!! If you’re interested in open source ML model monitoring, give this episode a listen🎧 Thank you

⚡️ twitter.com/OssStartup/sta

This Tweet is unavailable.

Feb 4, 2022

Translating users' needs into a great product is hard. Seems even harder for dev tools. You deal with so many layers of abstraction and infrastructure and have to make a bunch of calls re: architecture quite early still. Great thread 👇🏼

Quote Tweet

Yaroslav Bulatov

@yaroslavvb

Feb 4, 2022

Bananas and ML infrastructure: I've asked around about cloud workflows, and most of the feedback had unhappiness with cloud tooling. This prompted a discussion in @chipro's MLops community -- why are MLops frameworks so bad? (1/9)

Show this thread

Feb 2, 2022

ML monitoring is still is in its nascency. As a community, we are yet to define the right approaches to do it at scale. Something we hope to solve with

, and a big reason why we do it in open-source. A great overview of challenges by

@sh_reya

👇🏼

Quote Tweet

Shreya Shankar

@sh_reya

Jan 31, 2022

Over the last year, many people have told me that operationalizing machine learning isn’t a research problem. I disagree. In the final post of my ML monitoring series, I outline research challenges and solution ideas: shreya-shankar.com/rethinking-ml-

Jan 21, 2022

Awesome OSS integration to detect drift in ML pipelines:

@zenml_io

Alex Strick van Linschoten

❤️ So happy to see it live! We must have discussed the idea with

@htahir111

when both tools were about 2 months old. Thanks so much to

@strickvl

for making it happen and for a very kind blog:

Quote Tweet

@strickvl

Jan 21, 2022

A little love letter to a tool I've spent some time getting to know over the past few weeks in order to write an integration for @zenml_io. @EvidentlyAI is a monitoring tool for machine learning workflows. blog.zenml.io/zenml-loves-ev

Show this thread

Thrilled to see

on the list of fastest-growing open-source companies 🥳

Quote Tweet

Runa Capital

@RunaCapital

Jan 17, 2022

As usual, we are pleased to present the latest update to our ROSS index — the top-20 fastest-growing open-source companies in Q4 2021: runacap.com/ross-index/q4-

Show this thread

Jan 10, 2022

Absolutely awesome news especially for these 2-3 people companies where YC is the first investor.

Quote Tweet

Y Combinator

@ycombinator

Jan 10, 2022

We're excited to announce our new standard deal at Y Combinator. When a company is accepted into YC, we now invest a total of $500,000. @gralston shares more on our blog: blog.ycombinator.com/ycs-standard-d

Dec 31, 2021

📈 This year has been eventful for Evidently AI. We started 2021 as a bootstrapped team of two and a small Python library. We finish the year as a team of five, a YC company, and a comprehensive tool for ML model evaluation and monitoring. Look what happened in this recap👇🏼

Quote Tweet

Dec 31, 2021

2021 has been an incredible year for Evidently! We got into Y Combinator, shipped a lot, hired the team, and (almost) hit the 2000 GitHub stars. Read the recap here: evidentlyai.com/blog/2021-reca Wish you all a very Happy New Year!

Paige Bailey

@DynamicWebPaige

Dec 23, 2021

📊 Did you know that you can display

drift and model performance evaluations in

@code

? This even works in

@GitHub

Codespaces, exactly as you would expect in a

@ProjectJupyter

notebook. All you need to do is specify mode='inline' in .show(): github.com/evidentlyai/ev

GIF

Before starting to work on

, we spoke with 50+ data science teams and came to the same conclusion. The ML models in production most often use tabular data and don't use deep learning. Text comes second.

Quote Tweet

Bojan Tunguz

@tunguz

Text is slowly gaining more prominence in *conjunction* with relational data in the day-job of many Data Scientists. However, tabular data remains the cornerstone of what Data Science is still about in most work environments. 4/4

Show this thread

🔥 Our latest collaboration! Love this feature at

@DeepnoteHQ

to combine code and text in one interactive article:

Quote Tweet

A new collaboration:

Evidently reports +

@DeepnoteHQ notebooks! Check out this Deepnote project to: - read more about drift detection - see how to use Evidently in Deepnote - directly play with the code

deepnote.com/@EvidentlyAI/E

A new collaboration: 📊 Evidently reports + 📖

@DeepnoteHQ

notebooks! Check out this Deepnote project to: - read more about drift detection - see how to use Evidently in Deepnote - directly play with the code 👇🏼 deepnote.com/@EvidentlyAI/E

Dec 13, 2021

We are hosting the first Evidently community call this week! 👇🏼

Quote Tweet

Dec 13, 2021

Customize it! A major update: you can easily add custom tests, metrics, and components to the Evidently reports. Join the Community Call this Thursday to learn more: eventbrite.com/e/evidently-co More details in the release blog post: evidentlyai.com/blog/evidently

Dec 8, 2021

Awesome OSS ML news 👇🏼

Quote Tweet

deepset

@deepset_ai

Dec 8, 2021

1/3 Big news!! After more than 750 pull requests from 89 contributors and 19 months since our first release in May 2020 we are happy to announce the 1.0 release of #Haystack!

buff.ly/3rMrZYo (release notes here: buff.ly/3lLWEBs)

Show this thread

Nov 30, 2021

🥳 Evidently is turning one year today! Thanks to everyone who is using, testing, and supporting the project! Look how we grew 👇🏼

Khuyen Tran

@KhuyenTran16

Nov 12, 2021

Data drift is unexpected changes in model input data that can lead to model performance degradation. Thus, it is important to detect data drift when it occurs. Evidently allows you to do exactly in a few lines of # Python code.

GIF

Replying to

1. Gross underestimation of the expense & effort of ML projects. many believe a single DS or tool can save you. 2. Not defining problems correctly or aligning to business outcomes. 3. Gaslighting of DS that they need to be “full stack” and master the entire surface area 1/

121

Nov 9, 2021

Drift can happen without outliers, and outliers can appear without data drift. It might make sense to monitor both, with different workflows. 📖 Read on for a complete blog post: evidentlyai.com/blog/ml-monito 💬Join our Discord to ask your question: discord.com/invite/xZjKRaN (4/4)

With outlier detection, we look at the individual unusual inputs. Our goal is to decide that the model will handle them well. If we detect an outlier, we often send it for manual processing. We want this check to be sensitive to detect a single anomaly. (3/4)

With data drift, we look at the "global" data distributions. Our goal is to decide if we can still trust that the model. If we detect drift, model retraining is often the ideal action. We want this check to be robust to outliers and only react to meaningful changes. (2/4)

We've been experimenting with a new blog format. A summary write-up to an applied ML question we get often. Here is the first one. What is the difference between data drift and outlier detection for production ML models? (1/4)