使用 s2i 打包镜像

2021-02-28 12:26:19 ⋅ 2669 ⋅ 0 ⋅ 0

一、安装

Source-to-Image (S2I) is a toolkit and workflow for building reproducible container images from source code. S2I produces ready-to-run images by injecting source code into a container image and letting the container prepare that source code for execution. By creating self-assembling builder images, you can version and control your build environments exactly like you use container images to version your runtime environments.

s2i官网：https://github.com/openshift/source-to-image

1、下载对应的版本

wget https://github.com/openshift/source-to-image/releases/download/v1.3.1/source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz

2、解压缩

tar -xvf source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz

查看：

[root@ks-allinone s2i]# pwd
/root/s2i
[root@ks-allinone s2i]# ls -l
total 24068
-rwxr-xr-x 1 root root 17949824 Oct 10 02:40 s2i
-rw-r--r-- 1 root root  6692560 Oct 10 02:45 source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz
lrwxrwxrwx 1 root root        3 Oct 10 02:40 sti -> s2i
[root@ks-allinone s2i]#

3、加入到PATH路径

cd /root/s2i
cp s2i /usr/local/bin

查看

[root@ks-allinone s2i]# cd /usr/local/bin
[root@ks-allinone bin]# ls -l
total 391620
-rwxr-xr-x 1 root root  35861536 Feb 24 01:32 calicoctl
-rwxr-xr-x 1 root root       220 Feb 24 01:40 calicoctl.sh
-rwxr-x--- 1 root root       344 Feb 24 01:39 etcd
-rwxr-xr-x 1 root root  15246720 Mar 30  2018 etcdctl
drwx------ 2 root root      4096 Feb 24 01:39 etcd-scripts
-rwxr-xr-x 1 root root  41819776 Feb 24 01:30 helm
-rwxr-xr-x 1 root root  95601713 Feb 24 01:29 istioctl
-rwxr-xr-x 1 root root  40480768 Feb 24 01:26 kubeadm
-rwxr-xr-x 1 root root  42893312 Feb 24 01:27 kubectl
-rwxr-xr-x 1 root root 111135384 Feb 24 01:31 kubelet
drwxr-xr-x 2 kube root      4096 Feb 24 01:23 kubernetes-scripts
-rwxr-xr-x 1 root root  17949824 Feb 28 11:56 s2i
[root@ks-allinone bin]#

二、使用

打包方法：

Examples:

# Build a Docker image from a remote Git repository
$ s2i build https://github.com/openshift/ruby-hello-world centos/ruby-22-centos7 hello-world-app

# Build from a local directory.  If this directory is a git repo then the current commit will be built.
$ s2i build . centos/ruby-22-centos7 hello-world-app

示例

$ s2i build https://github.com/sclorg/django-ex centos/python-35-centos7 hello-python

在上述s2i build命令中，我们选定一个builder构建器，其基于镜像centos/python-35-centos7，指定了存放在github仓库中（地址为https://github.com/sclorg/dja...）的源码路径，命令最终为我们生成了可运行的镜像，其为我们隐藏了构建镜像的细节，这些细节与操作交由构建器来实施。

查看打包的镜像：

[root@ks-allinone images]# docker images | grep 'python'
hello-python                                                   latest              23ac6f9bb627        8 minutes ago       675MB
centos/python-35-centos7                                       latest              2db34dda8fd8        23 months ago       645MB

启动镜像：

$ docker run -p 8080:8080 hello-python

三、实战

将模型包上传到服务器，然后使用打包命令对对这个模型包进行镜像打包。

打包镜像

[root@ks-allinone model_packages]# pwd
/root/model_packages
[root@ks-allinone model_packages]# ls -l
total 4
drwxr-xr-x 4 root root 4096 Feb 28 12:33 model_1

# 本地构建
 [root@ks-allinone model_packages]# s2i build ./model_1 centos/python-35-centos7 hello-model
---> Installing application source ...
Build completed successfully
[root@ks-allinone model_packages]# ls -l

查看已经打包的镜像：

[root@ks-allinone model_packages]# docker images | grep 'model'
hello-model                                                    latest              7a91ec13de40        52 seconds ago      648MB
[root@ks-allinone model_packages]#

You've just built and run a new container image from source code in a git repository, no Dockerfile necessary.

四、打包模型服务

1、镜像拉取

在上边安装好 s2i 后，再下载 seldon-core-s2i 基础镜像：

$ sudo docker pull seldonio/seldon-core-s2i-python3:0.18

2、上传模型包

模型包seldon-core-cll 上传到服务器/home/hemei/model/ 目录下，模型包文件列表：

[hemei@master seldon-core-cll]$ pwd
/home/hemei/model/seldon-core-cll
[hemei@master seldon-core-cll]$ ls -al
total 36
drwxrwxr-x. 3 hemei hemei   109 Feb 26 17:02 .
drwxrwxr-x. 9 hemei hemei   262 Feb 26 09:29 ..
-rw-rw-r--. 1 hemei hemei   873 Feb 25 15:54 finalized_model2.sav
-rw-rw-r--. 1 hemei hemei  1008 Feb 25 16:58 MyModel.py
-rw-rw-r--. 1 hemei hemei 24048 Feb 25 15:54 pima_data.csv
-rw-rw-r--. 1 hemei hemei    36 Feb 25 16:19 requirements.txt

2、模型包添加环境变量

在模型包中文件中添加环境变量文件 .s2i/environment

[hemei@master seldon-core-cll]$ ls -al
total 36
drwxrwxr-x. 3 hemei hemei   109 Feb 26 17:02 .
drwxrwxr-x. 9 hemei hemei   262 Feb 26 09:29 ..
-rw-rw-r--. 1 hemei hemei   873 Feb 25 15:54 finalized_model2.sav
-rw-rw-r--. 1 hemei hemei  1008 Feb 25 16:58 MyModel.py
-rw-rw-r--. 1 hemei hemei 24048 Feb 25 15:54 pima_data.csv
-rw-rw-r--. 1 hemei hemei    36 Feb 25 16:19 requirements.txt
drwxrwxr-x. 2 hemei hemei    25 Feb 26 19:37 .s2i
[hemei@master seldon-core-cll]$ cd .s2i/
[hemei@master .s2i]$ ls -l
total 4
-rw-rw-r--. 1 hemei hemei 66 Feb 25 16:04 environment
[hemei@master .s2i]$ cat environment 
MODEL_NAME=MyModel
API_TYPE=REST
SERVICE_TYPE=MODEL
PERSISTENCE=0
[hemei@master .s2i]$

模型包文件展示MyModel.py：

from pandas import read_csv
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from pandas import read_csv
from pickle import load
class MyModel:
    def __init__(self):
        filename = 'pima_data.csv'
        names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
        data = read_csv(filename, names=names)
        # 将数据分为输入数据和输出结果
        array = data.values
        X = array[:, 0:8]
        Y = array[:, 8]
        test_size = 0.33
        seed = 4
        X_train, X_test2, Y_traing, Y_test2 = train_test_split(X, Y, test_size=test_size, random_state=seed)
        model_file = 'finalized_model2.sav'
        with open(model_file, 'rb') as model_f:
           self._model = load(model_f)
        self._X_test2 = X_test2
        self._Y_test2 = Y_test2

    def predict(self,X,y,sample_weight=None):
        result = self._model.score(self._X_test2, self._Y_test2)
        return [result]

依赖文件：requirements.txt

scikit-learn==0.24.1
pandas==0.23.4

3、通过s2i打包镜像

sudo /usr/local/bin/s2i build /home/hemei/model/seldon-core-cll seldonio/seldon-core-s2i-python3:0.18  model-app-demo

查看打包好的镜像：

[hemei@master seldon-core-cll]$ sudo docker images | grep 'model'
[sudo] password for hemei: 
model-app-demo                                                         latest                     127a16149b34   52 minutes ago   1.07GB

此时的镜像可通过docker运行，也可做为seldonDepartment的镜像通过k8s部署

4、运行镜像

sudo docker run -p 5001:5000 model-app-demo

5、测试


[hemei@master ~]$ curl -d 'json={"data":{"ndarray":[[1.0,2.0,3.0,4.0]]}}' http://0.0.0.0:5001/predict
{"data":{"names":[],"ndarray":[0.8070866141732284]},"meta":{}}

6、重定向pip源

由于seldon-core-s2i-python3:0.18 中默认的pip源地址是国外的地址，下载缓慢。
这边使用自己重写的s2i 脚本，达到重新定向pip源的目的。
首先拿到基础镜像里面的assemble 脚本，添加以下配置pip源的脚本

file

其次在模型包环境配置的文件中添加源配置
file

最后s2i 创建镜像时指定s2i脚本目录
file

为者常成，行者常至

使用 s2i 打包镜像

一、安装

二、使用

三、实战

四、打包模型服务

1、镜像拉取

2、上传模型包

2、模型包添加环境变量

3、通过s2i打包镜像

4、运行镜像

5、测试

6、重定向pip源

AI

作者：Corwien

专栏推荐

使用 s2i 打包镜像

一、安装

二、使用

三、实战

四、打包模型服务

1、镜像拉取

2、上传模型包

2、模型包添加环境变量

3、通过s2i打包镜像

4、运行镜像

5、测试

6、重定向pip源

添加附言

AI

作者：Corwien

专栏推荐