~zod is running into some memory issues. trying some hotfixes to get it to turn over so we can find where we're leaking.
-
Show this thread
-
we hit a bail:meme, which is urbit for "no more memory." right now, we're trying to replay the events past the bail so that we can clear some cache or otherwise free up some space.
1 reply 0 retweets 2 likesShow this thread -
offending line, for those following along at home https://github.com/urbit/urbit/blob/a42f2cbe5ccc591c148464444264c8f6c92776e3/noun/manage.c#L39 …
1 reply 0 retweets 4 likesShow this thread -
we got ‘em back up enough to wipe the %ford cache and clear some space; can replay events now. networking off, of course, so we don’t get “neighbored” to death.
1 reply 0 retweets 0 likesShow this thread -
we’ll do some forensics on a copy of the pier to see what we can see and get ourselves ready for...
1 reply 0 retweets 0 likesShow this thread -
putting the galaxy back on the network
2 replies 0 retweets 3 likesShow this thread -
challenge now is resurrecting https://github.com/urbit/arvo/blob/master/gen/hood/mass.hoon … from its watery grave so we can see which vane is to blame
1 reply 0 retweets 2 likesShow this thread -
|mass is sort of back. now we can see who’s taking up all the space.pic.twitter.com/VlsSToR3nd
1 reply 0 retweets 2 likesShow this thread -
interesting results! the nock->bytecode cache and the ford build history/cache were the two big culprits. ford weighing in at ~500MB. doing some cleanup and will retry on a beefier instance.
1 reply 0 retweets 4 likesShow this thread -
well. after our emergency field surgery we got ~zod on the network...for one event.
1 reply 0 retweets 4 likesShow this thread
~zod had too many friends and tried to call them all back in the same event!
-
-
we have a better idea now of a way to clear our caches. well give it a shot and update here!
1 reply 0 retweets 3 likesShow this thread -
Two new PRs to address this issue: https://github.com/urbit/urbit/pull/1168 … andhttps://github.com/urbit/urbit/pull/1167 …
0 replies 0 retweets 3 likesShow this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.