How Shopify Built A Machine Learning Platform That Encourages Experimentation

SummaryShopify uses machine learning to power multiple features in their platform. In order to reduce the amount of effort required to develop and deploy models they have invested in building an opinionated platform for their engineers. They have gone through multiple iterations of the platform and their most recent version is called Merlin. In this episode Isaac Vidas shares the use cases that they are optimizing for, how it integrates into the rest of their data platform, and how they have designed it to let machine learning engineers experiment freely and safely.AnnouncementsHello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery.Your host is Tobias Macey and today I'm interviewing Isaac Vidas about his work on the ML platform used by ShopifyInterviewIntroductionHow did you get involved in machine learning?Can you describe what Shopify is and some of the ways that you are using ML at Shopify? What are the challenges that you have encountered as an organization in applying ML to your business needs?Can you describe how you have designed your current technical platform for supporting ML workloads? Who are the target personas for this platform?What does the workflow look like for a given data scientist/ML engineer/etc.?What are the capabilities that you are trying to optimize for in your current platform? What are some of the previous iterations of ML infrastructure and process that you have built?What are the most useful lessons that you gathered from those previous experiences that informed your current approach?How have the capabilities of the Merlin platform influenced the ways that ML is viewed and applied across Shopify?What are the most interesting, innovative, or unexpected ways that you have seen Merlin used?What are the most interesting, unexpected, or challenging lessons that you have learned while working on Merlin?When is Merlin the wrong choice?What do you have planned for the future of Merlin?Contact Info@kazuaros on TwitterLinkedInkazuar on GitHubParting QuestionFrom your perspective, what is the biggest barrier to adoption of machine learning today?Closing AnnouncementsThank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.If you've learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story.To help other people find the show please leave a review on iTunes and tell your friends and co-workersLinksShopifyShopify MerlinVertex AIscikit-learnXGBoostRayPodcast.__init__ EpisodePySparkGPT-3ChatGPTGoogle AIPyTorchPodcast.__init__ EpisodeDaskModinPodcast.__init__ EpisodeFlinkData Engineering Podcast EpisodeFeast Feature StoreKubernetesThe intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0