Conversation

This Tweet was deleted by the Tweet author. Learn more

Replying to

@jasonbaumgartne

If you have the bandwidth, would love to know how it was sampled too! :D

Replying to

and

Ditto on this sampling info! We've been working on different ways to sample YouTube over the last couple of years and never fully resolved how representative it was

Replying to

and

It is extremely hard to get concrete numbers on just how many videos are uploaded on a per hour / day basis, etc. In fact, I can't even find a definitive source on how many total videos Youtube had up to a specific date.

Replying to

and

We had the same issue! The closest we could ever get to an estimate was via this paper conferences.sigcomm.org/imc/2011/docs/

Replying to

and

Thanks Megan! That is helpful. Also, did you see my tweet about the ID structure of Youtube ids? I'm not entirely convinced that their ids are truly random. If there was any structure to the ids, that might give clues as to the level of activity for a time range.

Replying to

and

That's interesting -- my assumption was that they're random. I think that's the assumption the aforementioned paper relies on as well... I think it's entirely possible that the IDs are randomly selected since the ID space is so large, but I have no clue if that's true...

Replying to

and

In my previous tweet, I showed that the 11 digit ids are a slightly modified base64 representation of a 64 bit integer. We know Twitter uses the Snowflake algo to embed time data and server/node and datacenter ids so I'm wondering if Youtube's ID scheme uses something similar.

Replying to

and

For me it makes sense to aggregate data on a channel level, a large-scale crawl could use "subscribed lists" and get the largest component of the node. A good place to start is social blade data. I crawled some (72M videos)

arxiv.org

YouNiverse: Large-Scale Channel and Video Metadata from...

YouTube plays a key role in entertaining and informing people around the globe. However, studying the platform is difficult due to the lack of randomly sampled data and of systematic ways to query...

Replying to

and

I have some ideas about what I would do differently if I was starting again! Especially now that I have finished a first research project with this data... Would be keen to share :)