Project Overview

• Kakao i Machine Learning is an MLOps platform that can manage the entire process from AI model training to inference and production model deployment. It is composed of various components that support ML workflows such as data preparation, model development, and deployment.
• Users can configure the system to suit their environment by combining the components provided by Kakao i Machine Learning.
• Referenced Article

What did I do

MLOps Platform Development

I developed and operated the Kakao i Machine Learning MLOps platform.

• Participated in initial planning of the platform

• Participated in the initial planning of the MLOps platform by benchmarking Kubeflow, AzureML, Sagemaker, GCP, etc.

• Based on experience in the AI Lab Voice team, I actively expressed opinions on the usability of the platform from the user's point of view and contributed to the development of the platform.

• Development of Inference functionality

  • Developed the functionality to inference the trained model and external models in the MLOps platform.
  • Implemented the functionality in the API Server using Golang to support Triton, TFServe, Torch Serve, and Custom Model serving.
  • Implemented the functionality to wrap Yaml on the API Server so that the framework can be easily deployed on Kubernetes, and added the ability to use features such as Liveness and Readiness.
  • Implemented the functionality to support the gRPC protocol in addition to the existing HTTP protocol.

• Model Artifact resource functionality implementation

  • Implemented CRUD API for Model Artifact that represents the completed results in the MLOps platform.
  • Implemented the ability to add Meta information for Model Versioning.
  • Implemented the Tag option for the Model.

• Development of Managed Jupyter Notebook functionality

  

  

  • Implemented the Notebook component using Notebook open source components so that the Notebook function can be used in the MLOps platform.

  • Added the ability to dynamically change the Kernel Instance Type in the Notebook using the Enterprise Gateway and JupyterHub components.

  • 🌟 Kubeflow Open Source Contribute🌟

    • Contributed to Kubeflow Open Source by benchmarking the Kubeflow notebook controller to add Jupyter Notebook.

    • 관련 PR Link

    

• Development of Client functionality

  • Developed a Python Client (CLI, SDK) to allow API Server calls.
  • Developed Inference functionality and Model functionality Clients and made them easy to use by considering usability.

• Domain design for MLOps platform operation

  • Took the lead in deciding how to set the domain for each component, which components should be exposed externally, and how to manage the domains.
  • This made it easy for team members to understand and use domain naming.

• Additional functionality development for operation

  • Implemented the functionality to use GPU MIG resources in the MLOps platform.

• Establishment of API Server Error Code Policy

  • Established common error code policies for several API Servers and created error codes by reaching consensus among team members.
  • This contributed to smoother communication among teams such as frontend and planning teams.

  

Deployment process construction

I established and managed the overall deployment process of the service.