You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<title>From Manual to LLMs: Scaling Product Categorization - PyData Berlin 2025</title>
7
+
<metaname="description" content="How to use LLMs to categorize hundreds of thousands of products into 1,000 categories at scale? Learn about our journey from manual/rule-based methods, via fine... By Giampaolo Casolla, Ansgar Grüne">
<metaproperty="og:title" content="From Manual to LLMs: Scaling Product Categorization - PyData Berlin 2025">
13
+
<metaproperty="og:description" content="How to use LLMs to categorize hundreds of thousands of products into 1,000 categories at scale? Learn about our journey from manual/rule-based methods, via fine... By Giampaolo Casolla, Ansgar Grüne">
<metaname="twitter:title" content="From Manual to LLMs: Scaling Product Categorization - PyData Berlin 2025">
21
+
<metaname="twitter:description" content="How to use LLMs to categorize hundreds of thousands of products into 1,000 categories at scale? Learn about our journey from manual/rule-based methods, via fine... By Giampaolo Casolla, Ansgar Grüne">
<p>How to use LLMs to categorize hundreds of thousands of products into 1,000 categories at scale? Learn about our journey from manual/rule-based methods, via fine-tuned semantic models, to a robust multi-step process which uses embeddings and LLMs via the OpenAI APIs. This talk offers data scientists and AI practitioners learnings and best practices for putting such a complex LLM-based system into production. This includes prompt development, balancing cost vs. accuracy via model selection, testing mult-case vs. single-case prompts, and saving costs by using the OpenAI Batch API and a smart early-stopping approach. We also describe our automation and monitoring in a PySpark environment.</p>
148
+
149
+
150
+
<h3>Prerequisites</h3>
151
+
<p>Attendees should have a foundational understanding of machine learning concepts and familiarity with the Python data science stack. Exposure to vector embeddings or Large Language Model (LLM) APIs is helpful but not mandatory.</p>
152
+
153
+
154
+
<h2>Description</h2>
155
+
<divclass="description-content">
156
+
<p><strong>Target Audience:</strong> Data scientists, AI/ML engineers, and practitioners interested in applying large language models (LLMs) / generative AI to solve real-world classification problems at scale. Attendees should have a foundational understanding of machine learning concepts and familiarity with the Python data science stack. Exposure to vector embeddings or LLM APIs is helpful but not mandatory.</p>
157
+
<p><strong>Takeaway:</strong> Attendees will gain practical insights and learn best practices for building, debugging, scaling, and productionizing a complex, multi-step generative AI system for large-scale product categorization. They will understand the evolution from traditional methods to LLMs, learn specific techniques for prompt engineering, batch processing, cost optimization with models like OpenAI's, and see the tangible business impact of such a system.</p>
158
+
<p><strong>Detailed Outline:</strong></p>
159
+
<p>This talk chronicles our journey tackling a yet challenging problem: accurately categorizing hundreds of thousands of diverse products into a fine-grained taxonomy of over 1,000 categories. We'll share our evolution from initial manual and rule-based systems to a sophisticated, production-ready Generative AI pipeline.</p>
160
+
<ul>
161
+
<li><strong>Part 1: The Challenge & Initial Approaches (10 minutes)</strong><ul>
162
+
<li>Introduction to the business need for accurate product categorization at scale.</li>
163
+
<li>Overview of the limitations encountered with traditional methods:<ul>
164
+
<li>Manual Curation: Slow, expensive, inconsistent, and impossible to scale.</li>
165
+
<li>Rule-Based Systems: Brittle, hard to maintain, and unable to handle nuances or new product types.</li>
166
+
<li>Fine-tuned Semantic Models: An improvement, but struggled with zero-shot generalization to new categories and required significant labeled data and retraining.</li>
167
+
</ul>
168
+
</li>
169
+
</ul>
170
+
</li>
171
+
<li><strong>Part 2: Entering the GenAI Era - Iterations & Lessons Learned (10 minutes)</strong><ul>
172
+
<li>Our initial exploration using LLMs for categorization, what worked, and what failed.</li>
173
+
<li><strong>Developing the Prompt:</strong> We'll dive deep into the iterative process of prompt engineering for this complex multi-label, hierarchical classification task. We'll show examples of early prompts, their failure modes (e.g., inconsistent output format, hallucinated categories, difficulty handling multiple classification signals), and the refinements that led to more reliable results. We will discuss techniques for achieving structured output (e.g., JSON) from the LLM.</li>
174
+
<li><strong>Early Scaling Issues:</strong> Discussing the pitfalls of naive API usage, latency problems, and prohibitive costs when dealing with large volumes.</li>
175
+
</ul>
176
+
</li>
177
+
<li><strong>Part 3: Building a Robust, Scalable GenAI Pipeline (10 minutes)</strong><ul>
178
+
<li><strong>The Hybrid Approach:</strong> Detailing our successful multi-step architecture that combines the strengths of semantic embeddings for efficient candidate retrieval/filtering and LLMs (specifically leveraging OpenAI models) for nuanced final categorization.</li>
<li><em>Batching:</em> Implementing efficient batch processing using asynchronous requests and the OpenAI Batch API to drastically reduce latency and cost.</li>
181
+
<li><em>Cost vs. Accuracy:</em> Strategies for selecting the right model based on complexity and cost constraints.</li>
182
+
<li><em>Semantic Similarity & Early Stopping:</em> Using vector similarity to intelligently prune the search space, avoiding the need to evaluate every product against all 1,000+ categories with the LLM, thus significantly optimizing cost and throughput.</li>
183
+
<li><em>Automation & Monitoring</em>: How we process updates of categories and products automatically in PySpark and monitor that the live system works as expected.</li>
184
+
</ul>
185
+
</li>
186
+
</ul>
187
+
</li>
188
+
<li><strong>Part 4: Measuring Impact & Looking Ahead (10 minutes)</strong><ul>
189
+
<li>Presenting the results: Showcasing the significant improvements in categorization accuracy and coverage compared to previous methodsIllustrative examples of challenging products correctly categorized by the GenAI system.</li>
190
+
<li>Discussing the tangible business value derived as measured in A-B tests </li>
191
+
<li>Briefly touching upon ongoing work and future directions.</li>
192
+
</ul>
193
+
</li>
194
+
</ul>
195
+
<p>This presentation will focus on the practical application and engineering challenges, sharing reusable techniques and hard-won lessons applicable to anyone looking to leverage the power of generative AI for large-scale, real-world problems. We aim to provide a transparent account of not just the successes, but also the crucial learnings from failures encountered along the way.</p>
<p>Giampaolo Casolla is a Senior Data Scientist at GetYourGuide, leveraging advanced machine learning and Generative AI to solve complex travel industry challenges. With expertise spanning areas like Safety, Risk, and Security, and strong skills in stats, Python, R, and cloud tech, he brings a diverse background to the role. Prior to GetYourGuide, Giampaolo developed award-winning ML solutions at Amazon and has a background in research with publications and conference presentations. At GetYourGuide, he's focused on integrating LLMs and GenAI into data products to drive innovation in travel technology.</p>
<p>Ansgar Grüne is a Senior Data Scientist at GetYourGuide in Berlin. His work focuses on ML/AI approaches to improve the users search and discovery experience on the platform. He holds a Ph.D. in Theoretical Computer Science and has 10 years of experience as a Data Scientist in the travel industry following several years as software engineer.</p>
227
+
228
+
229
+
</div>
230
+
</div>
231
+
232
+
233
+
<divstyle="text-align: center; margin-top: 3em;">
234
+
<ahref="https://pydata.org/berlin2025/" class="button special">View Full Conference Program</a>
0 commit comments