: StatQuest,Andrew Karpathy, 3B1B on youtube. Start with basics and work your way towards transformers. For practical experience, try to build projects around LLM APIs (e.g., DeepSeek as it’s cheap, or OpenAI as it’s probably the easiest) with a focus on system design. You can also experiment with local models but they introduce their own set of problems and their application is more specific. Think of usecases you find interesting and build them. More on reddit.com

There is no one size fit all sequential order. and there will be an illusion of confidence if the fundamentals are not fully understood. I've only been working with ML for 3 years so take this with a grain of salt. Strang is the best you can get when it comes to linear algebra but I would not recommend it to a 12 year old https://youtu.be/QVKj3LADCnA watch through it 15 times. Jay https://jalammar.github.io/ provides great intuitive picture representations and easy explanations as well as providing supplemental resources but reading him alone you can fall into the illusion of confidence. The best I can do, for going through the whole shebang is Jeremy Howards Fast.ai courses. He will go through everything from math, the intuition, to programming fundamentals in an extremely digestible set of courses. He just create a stable diffusion course but I'm unsure how someone without the fundamentals will understand it. But It will take several months and you will fall into the dunning kruger effect and think you know what youre talking about. In reality, it will be like programming, it will take years to learn fully. But this is a great start. Andrew ng is great too, Steve Brunton, Machine learning street talk, https://www.youtube.com/@AndrejKarpathy course is starting to look great for understanding backprop. Finally, read papers and code as much as possible. Huggingface Deep RL course, AnthropicAi RLHF papers, Deepmind Flamingo, sparrow papers. Here is a great resource for beginning papers https://ml.berkeley.edu/reading-list/ and resources. You can most likely find a companion video on https://www.youtube.com/@YannicKilcher channel with great explanations. I hope I wasnt too much of a downer, I hope this helps. More on reddit.com

Thank you for the article! It was very informative. Would you describe the difference between Generative AI and Discriminative AI as similar to the difference between deduction and induction? More on reddit.com

Hey! I am a “Gen AI Engineer” :’) so i think i might be able to provide some guidance here. I’ve only talked about text models here. So: Learn about the attention mechanism. (No need to deep dive. Just understand what it does). Transformers vs RNNs vs LSTM/GRU (Again a brief overview should suffice). Different types of LLMs based on transformers. Encoder-Decoder, Decoder-Decoder, etc. Just skim through what types of architectures are popular LLMs such as GPT 3.5/4, Llama2, Mistral 7B or 8x7B based on. Open Source vs Closed Source LLMs: Which ones are better at the moment? Different companies involved in the LLM rat race such as OpenAI, Google DeepMind, Mistral, Anthropic, etc. How to access these? For open source explore platforms such as Huggingface and Ollama. Prompt Engineering: Get comfortable with writing prompts. I would suggest Andrew NGs short course on prompt engineering to understand methods such as few shot learning. Learn about each of these: What are tokens? What are Vector Embeddings and what are some popular embedding model available today?Why do we need VectorDBs such as FAISS, Pinecone or ChromaDB etc? What does context length of an LLM mean? What is Quantization of LLM weights? Difference between 4-bit, 8-bit, 16-bit LLMs. Retrieval Augmented Generation or RAG: Understand how training data used for LLMs might not have all the info you need, RAG allows you to perform question answering on your personal documents. At this point, you might want to explore frameworks such as Langchain anf LlamaIndex. These provide one stop solution for all GenAI related requirements of your application. Finetuning LLMs: Why do we need to finetune LLMs? How is it different from RAG? How much GPU memory/VRAM would I need to finetune a small LLM such as Llama2? Techniques such as LoRA, QLoRA, PEFT, DPO etc. Finetuning an LLM would require some understanding of frameworks such as Pytorch or tensorflow. Advanced features such as Agents, Tool use, Funtion calling, Multimodal LLMs, etc. Access various opensource models such from ollama or huggingface. Also get familiarized with using OpenAI’s API. I would also suggest try to work with streamlit. It’s a very convenient way of creating a frontend for your application. These were some points that i thought you might find useful. If you have any further questions, please feel free to reach out. More on reddit.com