www.icesr.com
IT运维工程师的摇篮

Docker: 使用jupyter notebook基础镜像搭建自己的 pytorch 开发环境

Docker: 使用jupyter notebook基础镜像搭建自己的PyTorch开发环境

关键字:docker jupyter notebook pytorch spotlight

原载地址:http://blog.csdn.net/jianchengss/article/details/78224778

启动最基本的jupyter notebook镜像:

使用基础镜像jupyter/datascience-notebook,因为它预装了常用的模块:pandas, matplotlib, scipy, seaborn, scikit-learn, scikit-image, sympy, cython, patsy, statsmodel, cloudpickle, dill, numba, bokeh

<code>docker <span class="hljs-command">run</span> -<span class="hljs-keyword">it</span> <span class="hljs-comment">--rm -p 8888:8888 jupyter/datascience-notebook:281505737f8a</span></code>

其中
docker run是使用一个镜像生成一个运行的容器;

-it指交互模式,启动后终端在运行着的容器里面,与之对应的有-d后端运行模式,启动后终端交互在实体机,要想进入容器需要使用命令docker exec -it container-name bash docker exec -it container-name意为交互模式进入正在运行的一个容器,bash意为进入容器后使用的命令,这里用的是bash,这样进入容器后就能执行shell;

--rm意为退出shell的时候自动删除容器,常在测试的时候使用,这样不用每次修改去删除已有的容器;

-p 8888:8888指的是端口映射,前面的是实体机的端口,后面是容器里面暴露出的端口,两边端口可以不一样,这样同一个镜像可以启动多个对应不同端口的服务;

jupyter/datascience-notebook:281505737f8a是镜像名字,冒号后面的是tag,类似于版本的概念,如果不显式的给出tag每次都回从hub上拉取latest的镜像,如果网络环境不好的话比较费时间,推荐显式给出tag,这样每次构建都会使用已有的镜像。

启动后就可以在终端看到:

<code>[I <span class="hljs-number">04</span>:<span class="hljs-number">01</span>:<span class="hljs-number">05.691</span> NotebookApp] Running the core application <span class="hljs-keyword">with</span> no additional extensions <span class="hljs-keyword">or</span> settings
[I <span class="hljs-number">04</span>:<span class="hljs-number">01</span>:<span class="hljs-number">05.692</span> NotebookApp] Serving notebooks from local directory: /home/jovyan
[I <span class="hljs-number">04</span>:<span class="hljs-number">01</span>:<span class="hljs-number">05.692</span> NotebookApp] <span class="hljs-number">0</span> active kernels
[I <span class="hljs-number">04</span>:<span class="hljs-number">01</span>:<span class="hljs-number">05.692</span> NotebookApp] The Jupyter Notebook <span class="hljs-keyword">is</span> running at:
[I <span class="hljs-number">04</span>:<span class="hljs-number">01</span>:<span class="hljs-number">05.692</span> NotebookApp] http://[<span class="hljs-keyword">all</span> ip addresses <span class="hljs-keyword">on</span> your system]:<span class="hljs-number">8888</span>/?token=<span class="hljs-number">0</span>a3331628e0e35f94eb0ad543faeb3e396fbccfa3ff06e5a
[I <span class="hljs-number">04</span>:<span class="hljs-number">01</span>:<span class="hljs-number">05.692</span> NotebookApp] <span class="hljs-keyword">Use</span> Control-C <span class="hljs-keyword">to</span> stop this server <span class="hljs-keyword">and</span> shut down <span class="hljs-keyword">all</span> kernels (twice <span class="hljs-keyword">to</span> skip confirmation).
[C <span class="hljs-number">04</span>:<span class="hljs-number">01</span>:<span class="hljs-number">05.692</span> NotebookApp] 

    Copy/paste this URL into your browser <span class="hljs-keyword">when</span> you connect <span class="hljs-keyword">for</span> the first <span class="hljs-typename">time</span>,
    <span class="hljs-keyword">to</span> login <span class="hljs-keyword">with</span> a token:
        http://localhost:<span class="hljs-number">8888</span>/?token=<span class="hljs-number">0</span>a3331628e0e35f94eb0ad543faeb3e396fbccfa3ff06e5a
</code>

此时是停在容器里面,打开浏览器 访问http://localhost:8888/?token=0a3331628e0e35f94eb0ad543faeb3e396fbccfa3ff06e5a 即可打开基本的jupyter notebook 环境,后面的token是随机生成的;

启动带权限的容器

生成自定义token

<code><span class="hljs-preprocessor"># Python脚本生成密码</span>
import IPython
IPython<span class="hljs-preprocessor">.lib</span><span class="hljs-preprocessor">.passwd</span>()</code>

输入密码生成token

<code><span class="hljs-tag">test</span>的<span class="hljs-tag">token</span>:<span class="hljs-tag">sha1</span><span class="hljs-pseudo">:6587feaef3b1</span><span class="hljs-pseudo">:6b243404e4cfaafe611fdf494ee71fdaa8c4a563</span></code>

自定义token运行容器:

<code>docker run <span class="hljs-attribute">-d</span> <span class="hljs-attribute">-p</span> <span class="hljs-number">8888</span>:<span class="hljs-number">8888</span> jupyter/datascience<span class="hljs-attribute">-notebook</span> start<span class="hljs-attribute">-notebook</span><span class="hljs-built_in">.</span>sh <span class="hljs-subst">--</span>NotebookApp<span class="hljs-built_in">.</span>password<span class="hljs-subst">=</span><span class="hljs-string">'sha1:6587feaef3b1:6b243404e4cfaafe611fdf494ee71fdaa8c4a563'</span></code>

这时访问http://localhost:8888/会出现输入密码的页面,输入正确的密码才能进入jupyter。

共享目录

-v参数

docker提供-v参数使实体机和容器共享目录,这对于有状态的服务很有用,目录挂载添加参数:
-v /home/jason/jason/docker/notebook:/home/jovyan/work

运行带有目录共享的容器

<code>docker run <span class="hljs-attribute">-it</span> <span class="hljs-subst">--</span>rm <span class="hljs-attribute">-p</span> <span class="hljs-number">8888</span>:<span class="hljs-number">8888</span> <span class="hljs-attribute">-v</span> /home/jason/jason/docker/notebook:/home/jovyan/work   jupyter/datascience<span class="hljs-attribute">-notebook</span> start<span class="hljs-attribute">-notebook</span><span class="hljs-built_in">.</span>sh <span class="hljs-subst">--</span>NotebookApp<span class="hljs-built_in">.</span>password<span class="hljs-subst">=</span><span class="hljs-string">'sha1:6587feaef3b1:6b243404e4cfaafe611fdf494ee71fdaa8c4a563'</span></code>

这样在jupyter里新建的notebook都会出现在实体机指定的目录里。由于这个镜像的原因 需在work目录下新建才能在实体机看到。

基于jupyter/datascience-notebook 生成pytorch image

Dockerfile

因为没有合适的pytorch镜像,自己编辑Dockerfile:

新建文件Dockerfile并编辑内容:

<code>FROM jupyter/datascience-notebook:<span class="hljs-number">281505737</span>f8a
MAINTAINER Jason.W. <span class="hljs-string">"jianchengss@163.com"</span>
<span class="hljs-preprocessor"># 下面是按官网的方法安装spotlight</span>
<span class="hljs-preprocessor">#RUN pip --no-cache-dir install --upgrade install http://download.pytorch.org/whl/cu75/torch-0.2.0.post3-cp36-cp36m-manylinux1_x86_64.whl </span>
<span class="hljs-preprocessor">#RUN pip --no-cache-dir install --upgrade torchvision</span>

<span class="hljs-preprocessor"># pytorch</span>
RUN conda install pytorch torchvision -c soumith
<span class="hljs-preprocessor"># spotlight(https://github.com/maciejkula/spotlight)</span>
RUN conda install -c maciejkula -c soumith spotlight=<span class="hljs-number">0.1</span><span class="hljs-number">.2</span></code>

build

在Dockerfile目录里运行命令:docker build -t jianchengss/datascience-pytorch:0.1 .
这样就生成了image:jianchengss/datascience-pytorch:0.1可以运行docker images查看本机上所有的image。

从构建的镜像运行容器

<code>docker run <span class="hljs-attribute">-it</span> <span class="hljs-subst">--</span>rm <span class="hljs-attribute">-p</span> <span class="hljs-number">8888</span>:<span class="hljs-number">8888</span> <span class="hljs-attribute">-v</span> ~/workspace/python/notebooks<span class="hljs-attribute">-pytorch</span>:/home/jovyan/work  <span class="hljs-subst">--</span>privileged<span class="hljs-subst">=</span><span class="hljs-literal">true</span> jianchengss/datascience<span class="hljs-attribute">-pytorch</span>:<span class="hljs-number">0.1</span> start<span class="hljs-attribute">-notebook</span><span class="hljs-built_in">.</span>sh <span class="hljs-subst">--</span>NotebookApp<span class="hljs-built_in">.</span>password<span class="hljs-subst">=</span><span class="hljs-string">'sha1:6587feaef3b1:6b243404e4cfaafe611fdf494ee71fdaa8c4a563'</span></code>

最终容器

经过以上步骤,测试完成后既可以执行最终运行的命令 注意 token换成自己的

<code>docker run <span class="hljs-attribute">-d</span> <span class="hljs-attribute">-p</span> <span class="hljs-number">8588</span>:<span class="hljs-number">8888</span> <span class="hljs-attribute">-v</span> ~/workspace/python/notebooks<span class="hljs-attribute">-pytorch</span>:/home/jovyan/work  <span class="hljs-subst">--</span>privileged<span class="hljs-subst">=</span><span class="hljs-literal">true</span> <span class="hljs-subst">--</span>name<span class="hljs-subst">=</span>pytorch jianchengss/datascience<span class="hljs-attribute">-pytorch</span>:<span class="hljs-number">0.1</span> start<span class="hljs-attribute">-notebook</span><span class="hljs-built_in">.</span>sh <span class="hljs-subst">--</span>NotebookApp<span class="hljs-built_in">.</span>password<span class="hljs-subst">=</span><span class="hljs-string">'sha1:7aee2f913c8e:17d40f203cbd5c9820f302894a92724c3de9fba6'</span>
</code>

-it --rm 换成了 -d,比之前多的参数有:

--name=pytorch,意为给container取一个名字,好区分和管理,缺省的话名字为一串随机的字符串。

--privileged=true出现文件夹访问权限的时候添加该属性

此时运行docker ps即可查看运行着的容器:

<code>CONTAINER ID        IMAGE                                 COMMAND                  CREATED             STATUS              PORTS                    NAMES
<span class="hljs-number">3</span>bd3e30e9ab3        jianchengss/datascience-pytorch:<span class="hljs-number">0.1</span>   <span class="hljs-string">"tini -- start-notebo"</span>   <span class="hljs-number">4</span> <span class="hljs-built_in">seconds</span> ago       Up <span class="hljs-number">3</span> <span class="hljs-built_in">seconds</span>        <span class="hljs-number">0.0</span><span class="hljs-number">.0</span><span class="hljs-number">.0</span>:<span class="hljs-number">8588</span>-&gt;<span class="hljs-number">8888</span>/tcp   pytorch
</code>

进入容器操作

容器启动后有时候需要进入容器操作,比方说查看信息或者安装新的软件,此时执行docker exec -it pytorch bash

其他命令

<code>docker stop container-<span class="hljs-property">name</span> <span class="hljs-comment"># 停止运行着的容器</span>
docker rm container-<span class="hljs-property">name</span> <span class="hljs-comment"># 删除已有的容器,要先停止</span>
docker rmi image-<span class="hljs-property">name</span> <span class="hljs-comment"># 删除已有的镜像</span></code>

未经允许不得转载:冰点网络 » Docker: 使用jupyter notebook基础镜像搭建自己的 pytorch 开发环境

分享到:更多 ()

评论 抢沙发

评论前必须登录!