The AI Canvas Newsletter #12

What's new with ML in production – Vicki Boykis

“In 2023, I wrote two pieces on machine learning engineering for The Pragmatic Programmer. (Part 1 and Part 2). However, since I started working with LLMs recently, neural architectures have changed some of those assumptions."

Sampling for Text Generation – Chip Huyen

“ML models are probabilistic. Imagine that you want to know what’s the best cuisine in the world. If you ask someone this question twice, a minute apart, their answers both times should be the same. If you ask a model the same question twice, its answer can change. If the model thinks that Vietnamese cuisine has a 70% chance of being the best cuisine and Italian cuisine has a 30% chance, it’ll answer “Vietnamese” 70% of the time, and “Italian” 30%. This probabilistic nature makes AI great for creative tasks. What is creativity but the ability to explore beyond the common possibilities, to think outside the box?”

Beware of misleading GPU vs CPU benchmarks – Itamar Turner-Trauring

“Do you use NumPy, Pandas, or scikit-learn and want to get faster results? Nvidia has created GPU-based replacements for each of these with the shared promise of extra speed. For example, if you visit the front page of NVidia’s RAPIDS project, you’ll see benchmarks showing cuDF, a GPU-based Pandas replacement, is 15× to 80× faster than Pandas! Unfortunately, while those speed-ups are impressive, they are also misleading. GPU-based libraries might be the answer to your performance problems… or they might be an unnecessary and expensive distraction.

LangGraph

“TL;DR: LangGraph is module built on top of LangChain to better enable creation of cyclical graphs, often needed for agent runtimes.”

Kindllm

“A distraction-free LLM chat web app optimized for Kindle. The perfect companion for your book. Powered by Mixtral from Mistral AI. Mainly tested on Kindle Paperwhites.”

Learning

Vector Databases: A Technical Primer - A collection of slides that give an excellent technical overview of vector databases.

Understanding Deep Learning - A free book from Simon J.D. Prince, covering the underlying ideas behind deep learning.

Machine Learning Engineering Open Book - An open collection of methodologies to help with successful training of large language models and multi-modal models.

Code

Instructor

“structured outputs for llms”

Data-drift

“Metrics Observability & Troubleshooting”

Vanna

“🤖 Chat with your SQL database 📊. Accurate Text-to-SQL Generation via LLMs using RAG 🔄.”

Nlm-ingestor

“This repo provides the server side code for llmsherpa API to connect. It includes parsers for various file formats.”

Business and Trends

• OpenAI CEO Sam Altman is still chasing billions to build AI chips
• Elon Musk’s AI startup that hopes to leapfrog OpenAI is halfway to its goal of landing $1 billion in funding
• Voice AI startup ElevenLabs gains unicorn status after latest fundraising -source
• Google DeepMind Scientists in Talks to Leave and Form AI Startup

Dive into 'The AI Canvas', our podcast exploring the transformative potential of generative AI. Engage in fireside chats, case studies, and innovative discussions on AI’s impact on industries and creativity.