Further Readings
MLOps Component
From the components covered in MLOps Concepts, the following diagram illustrates them.
The technology stacks covered in Everyone's MLOps are as follows.
| | Storage | Minio | | | Data Processing | Apache Spark | | | Data Visualization | Tableau | | Workflow Mgmt. | Orchestration | Airflow | | | Scheduling | Kubernetes | | Security & Compliance | Authentication & Authorization | Ldap | | | Data Encryption & Tokenization | Vault | | | Governance & Auditing | Open Policy Agent |
As you can see, there are still many MLOps components that we have not covered yet. We could not cover them all this time due to time constraints, but if you need it, it might be a good idea to refer to the following open source projects first.
For details:
| Mgmt. | Component | Open Soruce |
|---|---|---|
| Data Mgmt. | Collection | Kafka |
| Validation | Beam | |
| Feature Store | Flink | |
| ML Model Dev. & Experiment | Modeling | Jupyter |
| Analysis & Experiment Mgmt. | MLflow | |
| HPO Tuning & AutoML | Katib | |
| Deploy Mgmt. | Serving Framework | Seldon Core |
| A/B Test | Iter8 | |
| Monitoring | Grafana, Prometheus | |
| Process Mgmt. | pipeline | Kubeflow |
| CI/CD | Github Action | |
| Continuous Training | Argo Events | |
| Platform Mgmt. | Configuration Mgmt. | Consul |
| Code Version Mgmt. | Github, Minio | |
| Logging | (EFK) Elastic Search, Fluentd, Kibana | |
| Resource Mgmt. | Kubernetes |