Is there an update on Rizler's progress? It really fits my use case niche. Individually querying 100000 users isn't exactly realistic with pushshift ¯\_(ツ)_/¯
Conversation
Replying to
And one other thing, do the utc_hour_of... parameters currently work? It doesn't seem to be limiting it for me
1
Replying to
utc_hour_of ? Sorry I have a lot of APIs -- which are you talking about? I don't remember making any such parameter.
1
Replying to
Utc_hour_of_week and utc_hour_of_day, I found them in some read the docs for api.pushshift, it might have been an old document though.
1
Replying to
Ah fair enough. One more thing, as you can probably tell I'm not exactly experienced with APIs, but is it possible to send concurrent pushshift requests in batches, of course respecting the rate limit?
1
1
Replying to
Yes you can. Usually you would use an async library or multi threading.
With that said, be careful because it's very easy to make mistakes when you get into multiple threads, etc.. :)
2
1
Replying to
With respect to the rate limit, if I make 120 async requests then wait a second until I submit another 120, irrespective of the number of completed requests, does this respect the rate limit?
1
Replying to
Each call goes towards the 120 total calls per minute so if you make 120 calls in one second you will probably end up getting blocked by the firewall. That's a lot of calls per second.
Replying to
Ah, I don't really have much context for the rate sorry. What would be a suitable amount in your opinion?
2
Replying to
You can make up to 120 requests a minute (two a second). You can make short bursts provided you remain in that range for the entire minute.
1