使用 s2i 打包镜像
一、安装
Source-to-Image (S2I) is a toolkit and workflow for building reproducible container images from source code. S2I produces ready-to-run images by injecting source code into a container image and letting the container prepare that source code for execution. By creating self-assembling builder images, you can version and control your build environments exactly like you use container images to version your runtime environments.
s2i官网:https://github.com/openshift/source-to-image
1、下载对应的版本
wget https://github.com/openshift/source-to-image/releases/download/v1.3.1/source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz
2、解压缩
tar -xvf source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz
查看:
[root@ks-allinone s2i]# pwd
/root/s2i
[root@ks-allinone s2i]# ls -l
total 24068
-rwxr-xr-x 1 root root 17949824 Oct 10 02:40 s2i
-rw-r--r-- 1 root root 6692560 Oct 10 02:45 source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz
lrwxrwxrwx 1 root root 3 Oct 10 02:40 sti -> s2i
[root@ks-allinone s2i]#
3、加入到PATH路径
cd /root/s2i
cp s2i /usr/local/bin
查看
[root@ks-allinone s2i]# cd /usr/local/bin
[root@ks-allinone bin]# ls -l
total 391620
-rwxr-xr-x 1 root root 35861536 Feb 24 01:32 calicoctl
-rwxr-xr-x 1 root root 220 Feb 24 01:40 calicoctl.sh
-rwxr-x--- 1 root root 344 Feb 24 01:39 etcd
-rwxr-xr-x 1 root root 15246720 Mar 30 2018 etcdctl
drwx------ 2 root root 4096 Feb 24 01:39 etcd-scripts
-rwxr-xr-x 1 root root 41819776 Feb 24 01:30 helm
-rwxr-xr-x 1 root root 95601713 Feb 24 01:29 istioctl
-rwxr-xr-x 1 root root 40480768 Feb 24 01:26 kubeadm
-rwxr-xr-x 1 root root 42893312 Feb 24 01:27 kubectl
-rwxr-xr-x 1 root root 111135384 Feb 24 01:31 kubelet
drwxr-xr-x 2 kube root 4096 Feb 24 01:23 kubernetes-scripts
-rwxr-xr-x 1 root root 17949824 Feb 28 11:56 s2i
[root@ks-allinone bin]#
二、使用
打包方法:
Examples:
# Build a Docker image from a remote Git repository
$ s2i build https://github.com/openshift/ruby-hello-world centos/ruby-22-centos7 hello-world-app
# Build from a local directory. If this directory is a git repo then the current commit will be built.
$ s2i build . centos/ruby-22-centos7 hello-world-app
示例
$ s2i build https://github.com/sclorg/django-ex centos/python-35-centos7 hello-python
在上述s2i build命令中,我们选定一个builder构建器,其基于镜像centos/python-35-centos7,指定了存放在github仓库中(地址为https://github.com/sclorg/dja...)的源码路径,命令最终为我们生成了可运行的镜像,其为我们隐藏了构建镜像的细节,这些细节与操作交由构建器来实施。
查看打包的镜像:
[root@ks-allinone images]# docker images | grep 'python'
hello-python latest 23ac6f9bb627 8 minutes ago 675MB
centos/python-35-centos7 latest 2db34dda8fd8 23 months ago 645MB
启动镜像:
$ docker run -p 8080:8080 hello-python
三、实战
将模型包上传到服务器,然后使用打包命令对对这个模型包进行镜像打包。
打包镜像
[root@ks-allinone model_packages]# pwd
/root/model_packages
[root@ks-allinone model_packages]# ls -l
total 4
drwxr-xr-x 4 root root 4096 Feb 28 12:33 model_1
# 本地构建
[root@ks-allinone model_packages]# s2i build ./model_1 centos/python-35-centos7 hello-model
---> Installing application source ...
Build completed successfully
[root@ks-allinone model_packages]# ls -l
查看已经打包的镜像:
[root@ks-allinone model_packages]# docker images | grep 'model'
hello-model latest 7a91ec13de40 52 seconds ago 648MB
[root@ks-allinone model_packages]#
You've just built and run a new container image from source code in a git repository, no Dockerfile necessary.
四、打包模型服务
1、镜像拉取
在上边安装好 s2i
后,再下载 seldon-core-s2i
基础镜像:
$ sudo docker pull seldonio/seldon-core-s2i-python3:0.18
2、上传模型包
模型包seldon-core-cll
上传到服务器/home/hemei/model/
目录下,模型包文件列表:
[hemei@master seldon-core-cll]$ pwd
/home/hemei/model/seldon-core-cll
[hemei@master seldon-core-cll]$ ls -al
total 36
drwxrwxr-x. 3 hemei hemei 109 Feb 26 17:02 .
drwxrwxr-x. 9 hemei hemei 262 Feb 26 09:29 ..
-rw-rw-r--. 1 hemei hemei 873 Feb 25 15:54 finalized_model2.sav
-rw-rw-r--. 1 hemei hemei 1008 Feb 25 16:58 MyModel.py
-rw-rw-r--. 1 hemei hemei 24048 Feb 25 15:54 pima_data.csv
-rw-rw-r--. 1 hemei hemei 36 Feb 25 16:19 requirements.txt
2、模型包添加环境变量
在模型包中文件中添加环境变量文件 .s2i/environment
[hemei@master seldon-core-cll]$ ls -al
total 36
drwxrwxr-x. 3 hemei hemei 109 Feb 26 17:02 .
drwxrwxr-x. 9 hemei hemei 262 Feb 26 09:29 ..
-rw-rw-r--. 1 hemei hemei 873 Feb 25 15:54 finalized_model2.sav
-rw-rw-r--. 1 hemei hemei 1008 Feb 25 16:58 MyModel.py
-rw-rw-r--. 1 hemei hemei 24048 Feb 25 15:54 pima_data.csv
-rw-rw-r--. 1 hemei hemei 36 Feb 25 16:19 requirements.txt
drwxrwxr-x. 2 hemei hemei 25 Feb 26 19:37 .s2i
[hemei@master seldon-core-cll]$ cd .s2i/
[hemei@master .s2i]$ ls -l
total 4
-rw-rw-r--. 1 hemei hemei 66 Feb 25 16:04 environment
[hemei@master .s2i]$ cat environment
MODEL_NAME=MyModel
API_TYPE=REST
SERVICE_TYPE=MODEL
PERSISTENCE=0
[hemei@master .s2i]$
模型包文件展示MyModel.py
:
from pandas import read_csv
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from pandas import read_csv
from pickle import load
class MyModel:
def __init__(self):
filename = 'pima_data.csv'
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
data = read_csv(filename, names=names)
# 将数据分为输入数据和输出结果
array = data.values
X = array[:, 0:8]
Y = array[:, 8]
test_size = 0.33
seed = 4
X_train, X_test2, Y_traing, Y_test2 = train_test_split(X, Y, test_size=test_size, random_state=seed)
model_file = 'finalized_model2.sav'
with open(model_file, 'rb') as model_f:
self._model = load(model_f)
self._X_test2 = X_test2
self._Y_test2 = Y_test2
def predict(self,X,y,sample_weight=None):
result = self._model.score(self._X_test2, self._Y_test2)
return [result]
依赖文件:requirements.txt
scikit-learn==0.24.1
pandas==0.23.4
3、通过s2i打包镜像
sudo /usr/local/bin/s2i build /home/hemei/model/seldon-core-cll seldonio/seldon-core-s2i-python3:0.18 model-app-demo
查看打包好的镜像:
[hemei@master seldon-core-cll]$ sudo docker images | grep 'model'
[sudo] password for hemei:
model-app-demo latest 127a16149b34 52 minutes ago 1.07GB
此时的镜像可通过docker运行,也可做为seldonDepartment的镜像通过k8s部署
4、运行镜像
sudo docker run -p 5001:5000 model-app-demo
5、测试
[hemei@master ~]$ curl -d 'json={"data":{"ndarray":[[1.0,2.0,3.0,4.0]]}}' http://0.0.0.0:5001/predict
{"data":{"names":[],"ndarray":[0.8070866141732284]},"meta":{}}
6、重定向pip源
由于seldon-core-s2i-python3:0.18 中默认的pip源地址是国外的地址,下载缓慢。
这边使用自己重写的s2i 脚本,达到重新定向pip源的目的。
首先拿到基础镜像里面的assemble 脚本,添加以下配置pip源的脚本
其次在模型包环境配置的文件中添加源配置
最后s2i 创建镜像时指定s2i脚本目录
为者常成,行者常至
自由转载-非商用-非衍生-保持署名(创意共享3.0许可证)