When I was running everything inside once process I was hitting a bunch of hurdles with connection limits and the power of only 1 process running my code.
Prikaži ovu nit
Took awhile to get here but distributed computing is pretty cool.
Problem: Scan 1000 blogs, parse posts for videos
Previously: 10m runtime (1 script: chunked promise queue)
Now 
: ~27s (4 functions ran highly parallel)
The power of lambdas running lambdas is crazy @zeithq
-
-
-
Switching to serverless functions has enabled me to do each piece of work in a different lambda. This means that my main function doesn't have to chunk up the work anymore. Each lambda only has one socket open for the connection so it's super quick.
Prikaži ovu nit -
Here is what the "flow of lambdas" looks like: call "/scan-blogs" => for each blog call "/scan-blog" (parallel) => for each post call "/scan-post" (parallel)
Prikaži ovu nit -
The result is: each call to "/scan-blog" takes a max of ~30s, and since everything is run in parallel it only takes around ~30s overall!
Prikaži ovu nit
Kraj razgovora
Novi razgovor -
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.