小白利用矿渣打造"全能"AI学习平台

近几年矿渣不少,大家买来一般做nas使用,这里利用矿渣在nas的docker环境下打造"全能"AI学习平台.在群晖以及unraid上的docker下实验通过,按理其他docker环境也是可以的.学习框架 PyTorch、TensorFlow 和 scikit-learn主要工具 numpy、pandas、scipy、matplotlib图像处理 opencv

搭建过程如下:

宿主机的IP地址为 192.168.88.17

1 下载datascience-notebook 命令: docker pull jupyter/datascience-notebook

2 安装并运行容器 命令: docker run -d --restart always --name datascience-notebook --hostname datascience-notebook -p 10000:8888 -e JUPYTER_ENABLE_LAB=yes -v /mnt/disk1/docker/Jupyter:/home/jovyan jupyter/datascience-notebook

查看输出日志

Executing the command: jupyter lab
[I 06:51:32.316 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.8/site-packages/jupyterlab
[I 06:51:32.317 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 06:51:32.326 LabApp] Serving notebooks from local directory: /home/jovyan
[I 06:51:32.326 LabApp] Jupyter Notebook 6.1.5 is running at:
[I 06:51:32.327 LabApp] http://datascience-notebook:8888/?token=7ca343d083ac8caaa8f07cadd335975c9d1d9f85fef35dab
[I 06:51:32.327 LabApp] or http://127.0.0.1:8888/?token=7ca343d083ac8caaa8f07cadd335975c9d1d9f85fef35dab
[I 06:51:32.327 LabApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 06:51:32.338 LabApp]

3 浏览器输入 http://192.168.88.17:10000/?token=7ca343d083ac8caaa8f07cadd335975c9d1d9f85fef35dab

4 进入对应目录 命令: cd /mnt/disk1/docker/Jupyter/work/conda_bag/

A. 将re.txt 拷贝到 /mnt/disk1/docker/Jupyter/work/conda_bag/

B. 利用 wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64/tensorflow-base-2.3.0-eigen_py38hb57a387_0.tar.bz2 下载 tensorflow-base

5 命令行进入容器内部 命令: docker exec -it datascience-notebook /bin/bash

6 安装所需包 命令: pip install -r re.txt -i https://mirrors.aliyun.com/pypi/simple

如果出现MemoryError 使用命令安装 pip install --no-cache-dir -r re.txt -i https://mirrors.aliyun.com/pypi/simple

或者只安装处此问题的包 pip install --no-cache-dir torch -i https://mirrors.aliyun.com/pypi/simple

7 利用conda安装python-graphviz 命令: conda install python-graphviz

8(实际上这步我是省略了的) 添加conda国内源 命令如下 : (这样后可以直接 conda install XXX 但是我这里还是不行 所以下载到本地了再来安装)

网上说这样设置后 conda install 会好使 但我这里还是不好使

conda config --show-sources

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/

conda config --add channels https://anaconda.mirrors.sjtug.sjtu.edu.cn/pkgs/main

conda config --add channels https://anaconda.mirrors.sjtug.sjtu.edu.cn/pkgs/free

conda config --add channels https://anaconda.mirrors.sjtug.sjtu.edu.cn/pkgs/mro

conda config --add channels https://anaconda.mirrors.sjtug.sjtu.edu.cn/pkgs/msys2

conda config --add channels https://anaconda.mirrors.sjtug.sjtu.edu.cn/pkgs/pro

conda config --add channels https://anaconda.mirrors.sjtug.sjtu.edu.cn/pkgs/r

conda config --set show_channel_urls yes

conda config --show-sources

9 利用conda本地安装tensorflow 命令: conda install --use-local tensorflow-base-2.3.0-eigen_py38hb57a387_0.tar.bz2

(记住一定要先安装了 conda install python-graphviz 了再这样本地安装tensorflow 如果先这样本地安装了tensorflow 会导致conda install python-graphviz 安装不上)

10 三个框架的版本

11 opencv 演示

12 graphviz 演示

总结

absl-py alembic astunparse brewer2mpl cftime d2lzh eofs fastcache future gast google-auth-oauthlib google-pasta googleapis-common-protos jieba Keras Keras-Preprocessing mxnet netCDF4 opencv-contrib-python opencv-python opt_einsum promise pyasn1 pyasn1-modules pydot-ng pyepsg pyproj pytorch-lightning qtconsole QtPy scratch sentencepiece termcolor torch torchtext torchvision tornado Werkzeug wrapt xarray

（祝各位值友学的开心 ... ）