Conversation

1/ Starting my road to learning about AI. I’ve done some courses here and there but I am planning a more consistent effort over the next few years. Mostly documenting my path for others and for fun. Often people suggest things I didn’t know which can be helpful.
79
3,310
2/ I am doing course.fast.ai again. I did part 1 in 2018 but binged the first 4 lessons fri-sun. Jeremy was kind enough to invite me to the 2022 version on going. Wait a few weeks before pursuing this which will show you new tools and a few new concepts.
4/ I’ve been messing around with prompts again in OpenAI playground with DALL-E and GPT-3 to experiment. Made some album art for a song I plan to release next month and prototyped a new feature for I hope we’ll ship later. Just keeping up the momentum/motivation.
5
104
6/ I was somewhat surprised how complicated getting a dev env was today. Lots of choices but it's all kind of expensive, running weird versions of pkgs, bespoke notebook UIs. I mostly just wanna use a local jupyter environment and connect to a GPU/macOS/not pay $1K/mo
4
111
7/ Trying Paperspace but their remote kernel environment in VS Code doesn't work. Might give up and just use the funky jupyter lab thing and get to work.
10
53
9/ Hands down this is the best experience: all your VS Code extensions (vim, copilot, etc.) all in one integrated IDE running in a browser using cloud compute. Don't bother with anything else.
Quote Tweet
Writing AI code from @MightyApp using GitHub Codespaces with a V100 GPU in the cloud. You can train, commit code, and build all from a single tab in your browser using VS Code. Browser = OS
Show this thread
Image
4
148
11/ Had a nice conversation with yesterday about a project for Mighty I want to work on. He helped me think a bit about what data to collect, what sort of approach to take, how to avoid getting too complicated, starting with small amounts of data, etc Thank you!
1
51
13/ Going to look for something to build now and step away from the lessons for a bit. For me, it’s a bit hard to internalize things without struggling a bit on my own and working on a toy app. If you have any interesting data sets, pls reply.
6
83
14/ Decided on a toy project for Mighty to do address bar prediction using my own data. Goal: given the fewest letters, can it predict where I’d want to go?
5
51
15/ Grabbing a whiskey and going to train my first baseline model using just my own data for now. More data will come in next week.
3
43
16/ Worked for ~3 hours last night doing an NLP approach for this address bar problem but realized after that it made no sense after training failed. I feel a little stupid since I am not sure how an LM would predict the URL fragment. Learning! Trying collab filtering today.
1
38
20/ IT WORKS (I think?) A lot more testing to validate but huzzah! It was quite confident that the other address bar results were not a match as well which is neat. A big thank you to helping me get unstuck.
Image
Image
1
46
21/ Tonight I am going to go deep on learning about evaluating validation sets and figure out how to do a custom split since I am quite skeptical of results I am getting. Today I got gradient accumulation and accuracy metrics working. Added new data. Huggingface has solid docs.
2
40
22/ Got my custom splits working, generating a new kind of data to improve the model, explore Huggingface Datasets, and refactored a bunch of code to experiment faster. Next: going to learn about how to avoid overfitting.
2
34
25/ More positively: data collection is going well. I have over 16K samples of real-world address bar results (all opt-in). Seems reasonable it’ll hit 100K in no time.
2
20
27/ Worked through lesson 8 with real production data: youtu.be/htiNBPxcXgo Built a tabular model. Getting faster. Trying to figure out how to do a multi-modal approach. Increasingly interested in understanding how to build nets from scratch to internalize the fundamentals.
3
39
28/ My weekend reading is mostly going to be taking an exploratory turn to learning and messing around with LSTMs/RNNs:
Image
5
33
30/ all I want to do is make everything from scratch to fully understand what’s going on kaparthy was right My study block tomorrow is this chapter:
Image
1
44
31/ Lately, I've been using my AI Twitter list at the end of each week to open all the interesting new research, videos, or links to new github repos. Then I read each one and go down a rabbit hole if there's anything I haven't encountered to explore. Timeboxed to 2 hours.
5
60
32/ I found chapter 17 in the fast ai book really valuable to build things from scratch. I wasn't totally sure I understood forward/backward methods in a model before that. Mostly preparing to build a PyTorch model from scratch that's multi-modal (tabular/NLP) for this next run.
3
40
Show replies