Ubuntu 16.04 下編譯安裝Faiss CPU版本
阿新 • • 發佈：2019-02-07
環境：unbuntu 16.04
安裝anaconda2
#下載
wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda2-4.3.0-Linux-x86_64.sh
#許可權
chmod +x Anaconda2-4.3.0-Linux-x86_64.sh
#安裝
bash Anaconda2-4.3.0-Linux-x86_64.sh安裝過程anaconda會問要不要加入環境變數選是不行就手動新增
#編輯 subl $HOME/.bashrc #新增 export PATH="$HOME/anaconda2/bin:$PATH" #啟用 source $HOME/.bashrc
然後檢查一下是不是能用
conda list裝openblas
#安裝openblasconda install openblas#建立軟鏈sudo ln -s $HOME/anaconda2/lib/libopenblas.so.0 /usr/lib/libopenblas.so.0
下Faiss
# 下載FAISS原始碼
git clone https://github.com/facebookresearch/faiss.git
# 進入FAISS原始碼目錄
cd faiss
# 根據系統配置編譯環境
cp example_makefiles/makefile.inc.Linux ./makefile.inc
這裡需要進到 faiss目錄下的makefile.inc裡修改linux的版本我這裡把centos註釋掉，取消ubuntu16的註釋
cd
cd faiss/
subl makefile.inc下面是修改的部分
# 2. Openblas # # The library contains both BLAS and Lapack. About 30% slower than MKL. Please see # https://github.com/facebookresearch/faiss/wiki/Troubleshooting#slow-brute-force-search-with-openblas # to fix performance problemes with OpenBLAS BLASCFLAGS=-DFINTEGER=int # This is for Centos: #BLASLDFLAGS?=/usr/lib64/libopenblas.so.0 # for Ubuntu 16: sudo apt-get install libopenblas-dev python-numpy python-dev BLASLDFLAGS?=/usr/lib/libopenblas.so.0 # 編譯 &測試BLAS案例 make tests/test_blas ./tests/test_blas
C++環境
#編譯安裝
make
#下載資料集
wget ftp://ftp.irisa.fr/local/texmex/corpus/sift.tar.gz
tar -xzvf sift.tar.gz
# 轉移資料集。
mv sift sift1M
# 編譯執行
make tests/demo_sift1M
./tests/demo_sift1Mpython環境
# 更改配置檔案
subl makefile.inc
大概118行 PYTHONCFLAGS 選項，替換如下：
PYTHONCFLAGS=-I/home/JIAO/anaconda2/include/python2.7/ -I/home/JIAO/anaconda2/lib/python2.7/site-packages/numpy/core/include/
# 編譯.
make py
#如果出現以下，說明上面的改PYTHONCFLAGS沒有改對
python/swigfaiss_wrap.cxx:154:21: fatal error: Python.h: 沒有那個檔案或目錄compilation terminated.
# 檢驗
python -c "import faiss"
Failed to load GPU Faiss: No module named swigfaiss_gpuFaiss falling back to CPU-only.Traceback (most recent call last): File "<string>", line 1, in <module> File "faiss.py", line 28, in <module> from swigfaiss import * File "swigfaiss.py", line 28, in <module> _swigfaiss = swig_import_helper() File "swigfaiss.py", line 24, in swig_import_helper _mod = imp.load_module('_swigfaiss', fp, pathname, description)ImportError: /home/JIAO/anaconda2/bin/../lib/libgomp.so.1: version `GOMP_4.0' not found (required by ./_swigfaiss.so)
conda install libgcc
#檢視anaconda2 GOMP版本
strings/home/JIAO/anaconda2/bin/../lib/libgomp.so.1 | grep GOMP
#會看到GOMP 1.0 GOMP2.0 GOMP3.0 沒有GOMP4.0
#檢視linux自帶的GOMP版本有GOMP4.0
sudofind / -name libgomp.so.1*
#將anaconda2的gomp刪除連結到系統自帶的gomp4.0
sudo rm -rf /home/JIAO/anaconda2/bin/../lib/libgomp.so.1
sudo ln -s /usr/lib/x86_64-linux-gnu/libgomp.so.1/home/JIAO/anaconda2/bin/..libgomp.so.1
#有時候會出現swigfaiss failed
make clean
cd faiss/python
git checkout swigfaiss_gpu_wrap.cxx swigfaiss_gpu.py swigfaiss_wrap.cxx swigfaiss.py
cd faiss/
make py
# 6.1、簡單測試.
python -c "import faiss, numpy
#下面這行錯誤不用管
Failed to load GPU Faiss: No module named swigfaiss_gpuFaiss falling back to CPU-only.
faiss.Kmeans(10, 20).train(numpy.random.rand(1000, 10).astype('float32'))"
#複雜測試.
export PYTHONPATH=.
mkdir tmp
python tests/demo_auto_tune.py
下面是除錯的完整過程
[email protected]:~$ cd 下載
[email protected]:~/下載$ ls
Anaconda3-5.0.1-Linux-x86_64.sh libcudnn.so.6
cuda sublime_text_3_build_3143_x64.tar.bz2
cuda_8.0.61_375.26_linux.run teamviewer
cudnn-8.0-linux-x64-v6.0.tgz teamviewer_13.0.6634_amd64.deb
libcudnn.so
[email protected]d-up-to-11:~/下載$ wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda2-4.3.0-Linux-x86_64.sh
--2018-02-08 14:07:21-- https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda2-4.3.0-Linux-x86_64.sh
正在解析主機 mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)... 101.6.6.177, 2402:f000:1:416:101:6:6:178
正在連線 mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)|101.6.6.177|:443... 已連線。
已發出 HTTP 請求，正在等待迴應... 200 OK
長度： 483530594 (461M) [application/octet-stream]
正在儲存至: “Anaconda2-4.3.0-Linux-x86_64.sh”
Anaconda2-4.3.0-Lin 100%[===================>] 461.13M 10.3MB/s in 44s
2018-02-08 14:08:06 (10.6 MB/s) - 已儲存 “Anaconda2-4.3.0-Linux-x86_64.sh” [483530594/483530594])
[email protected]:~/下載$
[email protected]:~/下載$ chmod +x Anaconda2-4.3.0-Linux-x86_64.sh
[email protected]:~/下載$
[email protected]:~/下載$ bash Anaconda2-4.3.0-Linux-x86_64.sh
Welcome to Anaconda2 4.3.0 (by Continuum Analytics, Inc.)
In order to continue the installation process, please review the license
agreement.
Please, press ENTER to continue
>>>
================
Anaconda License
================
Copyright 2016, Continuum Analytics, Inc.
All rights reserved under the 3-clause BSD License:
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of Continuum Analytics, Inc. nor the names of its
contributors may be used to endorse or promote products derived from this
software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
Do you approve the license terms? [yes|no]
>>> yes
Anaconda2 will now be installed into this location:
/home/JIAO/anaconda2
- Press ENTER to confirm the location
- Press CTRL-C to abort the installation
- Or specify a different location below
[/home/JIAO/anaconda2] >>>
PREFIX=/home/JIAO/anaconda2
installing: python-2.7.13-0 ...
installing: _license-1.1-py27_1 ...
installing: alabaster-0.7.9-py27_0 ...
installing: anaconda-client-1.6.0-py27_0 ...
installing: anaconda-navigator-1.4.3-py27_0 ...
installing: argcomplete-1.0.0-py27_1 ...
installing: astroid-1.4.9-py27_0 ...
installing: astropy-1.3-np111py27_0 ...
installing: babel-2.3.4-py27_0 ...
installing: backports-1.0-py27_0 ...
installing: backports_abc-0.5-py27_0 ...
installing: beautifulsoup4-4.5.3-py27_0 ...
installing: bitarray-0.8.1-py27_0 ...
installing: blaze-0.10.1-py27_0 ...
installing: bokeh-0.12.4-py27_0 ...
installing: boto-2.45.0-py27_0 ...
installing: bottleneck-1.2.0-np111py27_0 ...
installing: cairo-1.14.8-0 ...
installing: cdecimal-2.3-py27_2 ...
installing: cffi-1.9.1-py27_0 ...
installing: chardet-2.3.0-py27_0 ...
installing: chest-0.2.3-py27_0 ...
installing: click-6.7-py27_0 ...
installing: cloudpickle-0.2.2-py27_0 ...
installing: clyent-1.2.2-py27_0 ...
installing: colorama-0.3.7-py27_0 ...
installing: configobj-5.0.6-py27_0 ...
installing: configparser-3.5.0-py27_0 ...
installing: contextlib2-0.5.4-py27_0 ...
installing: cryptography-1.7.1-py27_0 ...
installing: curl-7.52.1-0 ...
installing: cycler-0.10.0-py27_0 ...
installing: cython-0.25.2-py27_0 ...
installing: cytoolz-0.8.2-py27_0 ...
installing: dask-0.13.0-py27_0 ...
installing: datashape-0.5.4-py27_0 ...
installing: dbus-1.10.10-0 ...
installing: decorator-4.0.11-py27_0 ...
installing: dill-0.2.5-py27_0 ...
installing: docutils-0.13.1-py27_0 ...
installing: entrypoints-0.2.2-py27_0 ...
installing: enum34-1.1.6-py27_0 ...
installing: et_xmlfile-1.0.1-py27_0 ...
installing: expat-2.1.0-0 ...
installing: fastcache-1.0.2-py27_1 ...
installing: flask-0.12-py27_0 ...
installing: flask-cors-3.0.2-py27_0 ...
installing: fontconfig-2.12.1-2 ...
installing: freetype-2.5.5-2 ...
installing: funcsigs-1.0.2-py27_0 ...
installing: functools32-3.2.3.2-py27_0 ...
installing: futures-3.0.5-py27_0 ...
installing: get_terminal_size-1.0.0-py27_0 ...
installing: gevent-1.2.1-py27_0 ...
installing: glib-2.50.2-1 ...
installing: greenlet-0.4.11-py27_0 ...
installing: grin-1.2.1-py27_3 ...
installing: gst-plugins-base-1.8.0-0 ...
installing: gstreamer-1.8.0-0 ...
installing: h5py-2.6.0-np111py27_2 ...
installing: harfbuzz-0.9.39-2 ...
installing: hdf5-1.8.17-1 ...
installing: heapdict-1.0.0-py27_1 ...
installing: icu-54.1-0 ...
installing: idna-2.2-py27_0 ...
installing: imagesize-0.7.1-py27_0 ...
installing: ipaddress-1.0.18-py27_0 ...
installing: ipykernel-4.5.2-py27_0 ...
installing: ipython-5.1.0-py27_0 ...
installing: ipython_genutils-0.1.0-py27_0 ...
installing: ipywidgets-5.2.2-py27_1 ...
installing: isort-4.2.5-py27_0 ...
installing: itsdangerous-0.24-py27_0 ...
installing: jbig-2.1-0 ...
installing: jdcal-1.3-py27_0 ...
installing: jedi-0.9.0-py27_1 ...
installing: jinja2-2.9.4-py27_0 ...
installing: jpeg-9b-0 ...
installing: jsonschema-2.5.1-py27_0 ...
installing: jupyter-1.0.0-py27_3 ...
installing: jupyter_client-4.4.0-py27_0 ...
installing: jupyter_console-5.0.0-py27_0 ...
installing: jupyter_core-4.2.1-py27_0 ...
installing: lazy-object-proxy-1.2.2-py27_0 ...
installing: libffi-3.2.1-1 ...
installing: libgcc-4.8.5-2 ...
installing: libgfortran-3.0.0-1 ...
installing: libiconv-1.14-0 ...
installing: libpng-1.6.27-0 ...
installing: libsodium-1.0.10-0 ...
installing: libtiff-4.0.6-3 ...
installing: libxcb-1.12-1 ...
installing: libxml2-2.9.4-0 ...
installing: libxslt-1.1.29-0 ...
installing: llvmlite-0.15.0-py27_0 ...
installing: locket-0.2.0-py27_1 ...
installing: lxml-3.7.2-py27_0 ...
installing: markupsafe-0.23-py27_2 ...
installing: matplotlib-2.0.0-np111py27_0 ...
installing: mistune-0.7.3-py27_0 ...
installing: mkl-2017.0.1-0 ...
installing: mkl-service-1.1.2-py27_3 ...
installing: mpmath-0.19-py27_1 ...
installing: multipledispatch-0.4.9-py27_0 ...
installing: nbconvert-4.2.0-py27_0 ...
installing: nbformat-4.2.0-py27_0 ...
installing: networkx-1.11-py27_0 ...
installing: nltk-3.2.2-py27_0 ...
installing: nose-1.3.7-py27_1 ...
installing: notebook-4.3.1-py27_0 ...
installing: numba-0.30.1-np111py27_0 ...
installing: numexpr-2.6.1-np111py27_2 ...
installing: numpy-1.11.3-py27_0 ...
installing: numpydoc-0.6.0-py27_0 ...
installing: odo-0.5.0-py27_1 ...
installing: openpyxl-2.4.1-py27_0 ...
installing: openssl-1.0.2k-0 ...
installing: pandas-0.19.2-np111py27_1 ...
installing: partd-0.3.7-py27_0 ...
installing: path.py-10.0-py27_0 ...
installing: pathlib2-2.2.0-py27_0 ...
installing: patsy-0.4.1-py27_0 ...
installing: pcre-8.39-1 ...
installing: pep8-1.7.0-py27_0 ...
installing: pexpect-4.2.1-py27_0 ...
installing: pickleshare-0.7.4-py27_0 ...
installing: pillow-4.0.0-py27_0 ...
installing: pip-9.0.1-py27_1 ...
installing: pixman-0.34.0-0 ...
installing: ply-3.9-py27_0 ...
installing: prompt_toolkit-1.0.9-py27_0 ...
installing: psutil-5.0.1-py27_0 ...
installing: ptyprocess-0.5.1-py27_0 ...
installing: py-1.4.32-py27_0 ...
installing: pyasn1-0.1.9-py27_0 ...
installing: pycairo-1.10.0-py27_0 ...
installing: pycosat-0.6.1-py27_1 ...
installing: pycparser-2.17-py27_0 ...
installing: pycrypto-2.6.1-py27_4 ...
installing: pycurl-7.43.0-py27_2 ...
installing: pyflakes-1.5.0-py27_0 ...
installing: pygments-2.1.3-py27_0 ...
installing: pylint-1.6.4-py27_1 ...
installing: pyopenssl-16.2.0-py27_0 ...
installing: pyparsing-2.1.4-py27_0 ...
installing: pyqt-5.6.0-py27_2 ...
installing: pytables-3.3.0-np111py27_0 ...
installing: pytest-3.0.5-py27_0 ...
installing: python-dateutil-2.6.0-py27_0 ...
installing: pytz-2016.10-py27_0 ...
installing: pyyaml-3.12-py27_0 ...
installing: pyzmq-16.0.2-py27_0 ...
installing: qt-5.6.2-3 ...
installing: qtawesome-0.4.3-py27_0 ...
installing: qtconsole-4.2.1-py27_1 ...
installing: qtpy-1.2.1-py27_0 ...
installing: readline-6.2-2 ...
installing: redis-3.2.0-0 ...
installing: redis-py-2.10.5-py27_0 ...
installing: requests-2.12.4-py27_0 ...
installing: rope-0.9.4-py27_1 ...
installing: scandir-1.4-py27_0 ...
installing: scikit-image-0.12.3-np111py27_1 ...
installing: scikit-learn-0.18.1-np111py27_1 ...
installing: scipy-0.18.1-np111py27_1 ...
installing: seaborn-0.7.1-py27_0 ...
installing: setuptools-27.2.0-py27_0 ...
installing: simplegeneric-0.8.1-py27_1 ...
installing: singledispatch-3.4.0.3-py27_0 ...
installing: sip-4.18-py27_0 ...
installing: six-1.10.0-py27_0 ...
installing: snowballstemmer-1.2.1-py27_0 ...
installing: sockjs-tornado-1.0.3-py27_0 ...
installing: sphinx-1.5.1-py27_0 ...
installing: spyder-3.1.2-py27_0 ...
installing: sqlalchemy-1.1.5-py27_0 ...
installing: sqlite-3.13.0-0 ...
installing: ssl_match_hostname-3.4.0.2-py27_1 ...
installing: statsmodels-0.6.1-np111py27_1 ...
installing: subprocess32-3.2.7-py27_0 ...
installing: sympy-1.0-py27_0 ...
installing: terminado-0.6-py27_0 ...
installing: tk-8.5.18-0 ...
installing: toolz-0.8.2-py27_0 ...
installing: tornado-4.4.2-py27_0 ...
installing: traitlets-4.3.1-py27_0 ...
installing: unicodecsv-0.14.1-py27_0 ...
installing: wcwidth-0.1.7-py27_0 ...
installing: werkzeug-0.11.15-py27_0 ...
installing: wheel-0.29.0-py27_0 ...
installing: widgetsnbextension-1.2.6-py27_0 ...
installing: wrapt-1.10.8-py27_0 ...
installing: xlrd-1.0.0-py27_0 ...
installing: xlsxwriter-0.9.6-py27_0 ...
installing: xlwt-1.2.0-py27_0 ...
installing: xz-5.2.2-1 ...
installing: yaml-0.1.6-0 ...
installing: zeromq-4.1.5-0 ...
installing: zlib-1.2.8-3 ...
installing: anaconda-4.3.0-np111py27_0 ...
installing: ruamel_yaml-0.11.14-py27_1 ...
installing: conda-4.3.8-py27_0 ...
installing: conda-env-2.6.0-0 ...
Python 2.7.13 :: Continuum Analytics, Inc.
creating default environment...
installation finished.
WARNING:
You currently have a PYTHONPATH environment variable set. This may cause
unexpected behavior when running the Python interpreter in Anaconda2.
For best results, please verify that your PYTHONPATH only points to
directories of packages that are compatible with the Python interpreter
in Anaconda2: /home/JIAO/anaconda2
Do you wish the installer to prepend the Anaconda2 install location
to PATH in your /home/JIAO/.bashrc ? [yes|no]
[no] >>> yes
Prepending PATH=/home/JIAO/anaconda2/bin to PATH in /home/JIAO/.bashrc
A backup will be made to: /home/JIAO/.bashrc-anaconda2.bak
For this change to become active, you have to open a new terminal.
Thank you for installing Anaconda2!
Share your notebooks and packages on Anaconda Cloud!
Sign up for free: https://anaconda.org
[email protected]:~/下載$ subl $HOME/.bashrc
[email protected]:~/下載$ source $HOME/.bashrc
[email protected]:~/下載$
[email protected]:~/下載$ conda list
# packages in environment at /home/JIAO/anaconda2:
#
_license 1.1 py27_1
alabaster 0.7.9 py27_0
anaconda 4.3.0 np111py27_0
anaconda-client 1.6.0 py27_0
anaconda-navigator 1.4.3 py27_0
argcomplete 1.0.0 py27_1
astroid 1.4.9 py27_0
astropy 1.3 np111py27_0
babel 2.3.4 py27_0
backports 1.0 py27_0
backports_abc 0.5 py27_0
beautifulsoup4 4.5.3 py27_0
bitarray 0.8.1 py27_0
blaze 0.10.1 py27_0
bokeh 0.12.4 py27_0
boto 2.45.0 py27_0
bottleneck 1.2.0 np111py27_0
cairo 1.14.8 0
cdecimal 2.3 py27_2
cffi 1.9.1 py27_0
chardet 2.3.0 py27_0
chest 0.2.3 py27_0
click 6.7 py27_0
cloudpickle 0.2.2 py27_0
clyent 1.2.2 py27_0
colorama 0.3.7 py27_0
conda 4.3.8 py27_0
conda-env 2.6.0 0
configobj 5.0.6 py27_0
configparser 3.5.0 py27_0
contextlib2 0.5.4 py27_0
cryptography 1.7.1 py27_0
curl 7.52.1 0
cycler 0.10.0 py27_0
cython 0.25.2 py27_0
cytoolz 0.8.2 py27_0
dask 0.13.0 py27_0
datashape 0.5.4 py27_0
dbus 1.10.10 0
decorator 4.0.11 py27_0
dill 0.2.5 py27_0
docutils 0.13.1 py27_0
entrypoints 0.2.2 py27_0
enum34 1.1.6 py27_0
et_xmlfile 1.0.1 py27_0
expat 2.1.0 0
fastcache 1.0.2 py27_1
flask 0.12 py27_0
flask-cors 3.0.2 py27_0
fontconfig 2.12.1 2
freetype 2.5.5 2
funcsigs 1.0.2 py27_0
functools32 3.2.3.2 py27_0
futures 3.0.5 py27_0
get_terminal_size 1.0.0 py27_0
gevent 1.2.1 py27_0
glib 2.50.2 1
greenlet 0.4.11 py27_0
grin 1.2.1 py27_3
gst-plugins-base 1.8.0 0
gstreamer 1.8.0 0
h5py 2.6.0 np111py27_2
harfbuzz 0.9.39 2
hdf5 1.8.17 1
heapdict 1.0.0 py27_1
icu 54.1 0
idna 2.2 py27_0
imagesize 0.7.1 py27_0
ipaddress 1.0.18 py27_0
ipykernel 4.5.2 py27_0
ipython 5.1.0 py27_0
ipython_genutils 0.1.0 py27_0
ipywidgets 5.2.2 py27_1
isort 4.2.5 py27_0
itsdangerous 0.24 py27_0
jbig 2.1 0
jdcal 1.3 py27_0
jedi 0.9.0 py27_1
jinja2 2.9.4 py27_0
jpeg 9b 0
jsonschema 2.5.1 py27_0
jupyter 1.0.0 py27_3
jupyter_client 4.4.0 py27_0
jupyter_console 5.0.0 py27_0
jupyter_core 4.2.1 py27_0
lazy-object-proxy 1.2.2 py27_0
libffi 3.2.1 1
libgcc 4.8.5 2
libgfortran 3.0.0 1
libiconv 1.14 0
libpng 1.6.27 0
libsodium 1.0.10 0
libtiff 4.0.6 3
libxcb 1.12 1
libxml2 2.9.4 0
libxslt 1.1.29 0
llvmlite 0.15.0 py27_0
locket 0.2.0 py27_1
lxml 3.7.2 py27_0
markupsafe 0.23 py27_2
matplotlib 2.0.0 np111py27_0
mistune 0.7.3 py27_0
mkl 2017.0.1 0
mkl-service 1.1.2 py27_3
mpmath 0.19 py27_1
multipledispatch 0.4.9 py27_0
nbconvert 4.2.0 py27_0
nbformat 4.2.0 py27_0
networkx 1.11 py27_0
nltk 3.2.2 py27_0
nose 1.3.7 py27_1
notebook 4.3.1 py27_0
numba 0.30.1 np111py27_0
numexpr 2.6.1 np111py27_2
numpy 1.11.3 py27_0
numpydoc 0.6.0 py27_0
odo 0.5.0 py27_1
openpyxl 2.4.1 py27_0
openssl 1.0.2k 0
pandas 0.19.2 np111py27_1
partd 0.3.7 py27_0
path.py 10.0 py27_0
pathlib2 2.2.0 py27_0
patsy 0.4.1 py27_0
pcre 8.39 1
pep8 1.7.0 py27_0
pexpect 4.2.1 py27_0
pickleshare 0.7.4 py27_0
pillow 4.0.0 py27_0
pip 9.0.1 py27_1
pixman 0.34.0 0
ply 3.9 py27_0
prompt_toolkit 1.0.9 py27_0
psutil 5.0.1 py27_0
ptyprocess 0.5.1 py27_0
py 1.4.32 py27_0
pyasn1 0.1.9 py27_0
pycairo 1.10.0 py27_0
pycosat 0.6.1 py27_1
pycparser 2.17 py27_0
pycrypto 2.6.1 py27_4
pycurl 7.43.0 py27_2
pyflakes 1.5.0 py27_0
pygments 2.1.3 py27_0
pylint 1.6.4 py27_1
pyopenssl 16.2.0 py27_0
pyparsing 2.1.4 py27_0
pyqt 5.6.0 py27_2
pytables 3.3.0 np111py27_0
pytest 3.0.5 py27_0
python 2.7.13 0
python-dateutil 2.6.0 py27_0
pytz 2016.10 py27_0
pyyaml 3.12 py27_0
pyzmq 16.0.2 py27_0
qt 5.6.2 3
qtawesome 0.4.3 py27_0
qtconsole 4.2.1 py27_1
qtpy 1.2.1 py27_0
readline 6.2 2
redis 3.2.0 0
redis-py 2.10.5 py27_0
requests 2.12.4 py27_0
rope 0.9.4 py27_1
ruamel_yaml 0.11.14 py27_1
scandir 1.4 py27_0
scikit-image 0.12.3 np111py27_1
scikit-learn 0.18.1 np111py27_1
scipy 0.18.1 np111py27_1
seaborn 0.7.1 py27_0
setuptools 27.2.0 py27_0
simplegeneric 0.8.1 py27_1
singledispatch 3.4.0.3 py27_0
sip 4.18 py27_0
six 1.10.0 py27_0
snowballstemmer 1.2.1 py27_0
sockjs-tornado 1.0.3 py27_0
sphinx 1.5.1 py27_0
spyder 3.1.2 py27_0
sqlalchemy 1.1.5 py27_0
sqlite 3.13.0 0
ssl_match_hostname 3.4.0.2 py27_1
statsmodels 0.6.1 np111py27_1
subprocess32 3.2.7 py27_0
sympy 1.0 py27_0
terminado 0.6 py27_0
tk 8.5.18 0
toolz 0.8.2 py27_0
tornado 4.4.2 py27_0
traitlets 4.3.1 py27_0
unicodecsv 0.14.1 py27_0
wcwidth 0.1.7 py27_0
werkzeug 0.11.15 py27_0
wheel 0.29.0 py27_0
widgetsnbextension 1.2.6 py27_0
wrapt 1.10.8 py27_0
xlrd 1.0.0 py27_0
xlsxwriter 0.9.6 py27_0
xlwt 1.2.0 py27_0
xz 5.2.2 1
yaml 0.1.6 0
zeromq 4.1.5 0
zlib 1.2.8 3
[email protected]:~/下載$ python
Python 2.7.13 |Anaconda 4.3.0 (64-bit)| (default, Dec 20 2016, 23:09:15)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>>
[1]+ 已停止 python
[email protected]:~/下載$ python --version
Python 2.7.13 :: Anaconda 4.3.0 (64-bit)
[email protected]:~/下載$
[email protected]:~/下載$ conda install openblas
Fetching package metadata ...
......
Solving package specifications: .
Package plan for installation in environment /home/JIAO/anaconda2:
The following NEW packages will be INSTALLED:
openblas: 0.2.19-0
The following packages will be UPDATED:
conda: 4.3.8-py27_0 --> 4.3.30-py27h6ae6dc7_0
Proceed ([y]/n)?
openblas-0.2.1 100% |################################| Time: 0:00:07 446.81 kB/s
conda-4.3.30-p 100% |################################| Time: 0:00:00 997.60 kB/s
[email protected]:~/下載$ ln -s $HOME/anaconda2/lib/libopenblas.so.0 /usr/lib64/libopenblas.so.0
ln: 無法建立符號連結'/usr/lib64/libopenblas.so.0': 沒有那個檔案或目錄
[email protected]:~/下載$
[email protected]:~/下載$ ln -s $HOME/anaconda2/lib/libopenblas.so.0 /usr/lib/libopenblas.so.0
ln: 無法建立符號連結'/usr/lib/libopenblas.so.0': 檔案已存在
[email protected]:~/下載$ cd
[email protected]:~$ git clone https://github.com/facebookresearch/faiss.git
正克隆到 'faiss'...
remote: Counting objects: 4924, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 4924 (delta 0), reused 1 (delta 0), pack-reused 4917
接收物件中: 100% (4924/4924), 8.17 MiB | 181.00 KiB/s, 完成.
處理 delta 中: 100% (4234/4234), 完成.
檢查連線... 完成。
[email protected]:~$
[email protected]:~$ cd faiss/
[email protected]:~/faiss$ cp example_makefiles/makefile.inc.Linux ./makefile.inc
[email protected]:~/faiss$
[email protected]:~/faiss$ make tests/test_blas
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp tests/test_blas.cpp -o tests/test_blas -g -fPIC -fopenmp /usr/lib64/libopenblas.so.0 -DFINTEGER=int
g++: error: /usr/lib64/libopenblas.so.0: 沒有那個檔案或目錄
Makefile:59: recipe for target 'tests/test_blas' failed
make: *** [tests/test_blas] Error 1
[email protected]:~/faiss$
[email protected]:~/faiss$ make tests/test_blas
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp tests/test_blas.cpp -o tests/test_blas -g -fPIC -fopenmp /usr/lib/libopenblas.so.0 -DFINTEGER=int
[email protected]:~/faiss$ ./tests/test_blas
BLAS test
errors=
0.000 0.000 0.000 0.000 0.000 0.000 0.000 -0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 -0.000 0.000 0.000 0.000 0.000 0.000 -0.000 0.000 0.000 0.000 0.000 -0.000 0.000 0.000 -0.000 -0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 -0.000 0.000 -0.000 -0.000 0.000 0.000 0.000 0.000 -0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 0.000 -0.000 0.000 -0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 0.000 -0.000 0.000 -0.000 0.000 -0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 -0.000
0.000 0.000 0.000 0.000 0.000 -0.000 0.000 0.000 0.000 -0.000 -0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 -0.000 0.000
0.000 0.000 -0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 -0.000 0.000 0.000 0.000 -0.000 0.000 0.000 0.000 0.000 -0.000
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 -0.000 -0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 -0.000 0.000 -0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 -0.000 -0.000 0.000 0.000 0.000 0.000 0.000
0.000 0.000 0.000 -0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 -0.000 0.000 0.000 0.000 0.000 0.000
Intentional Lapack error (appears only for 64-bit INTEGER):
info=0000064b00000000
Lapack uses 32-bit integers
[email protected]:~/faiss$
[email protected]:~/faiss$ make
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c hamming.cpp -o hamming.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c utils.cpp -o utils.o -DFINTEGER=int
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c IndexFlat.cpp -o IndexFlat.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c IndexIVF.cpp -o IndexIVF.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c IndexLSH.cpp -o IndexLSH.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c IndexPQ.cpp -o IndexPQ.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c IndexIVFPQ.cpp -o IndexIVFPQ.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c Clustering.cpp -o Clustering.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c Heap.cpp -o Heap.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c VectorTransform.cpp -o VectorTransform.o -DFINTEGER=int
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c index_io.cpp -o index_io.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c PolysemousTraining.cpp -o PolysemousTraining.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c MetaIndexes.cpp -o MetaIndexes.o
MetaIndexes.cpp:286:6: warning: ‘void faiss::{anonymous}::translate_labels(long int, faiss::{anonymous}::idx_t*, long int)’ defined but not used [-Wunused-function]
void translate_labels (long n, idx_t *labels, long translation)
^
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c Index.cpp -o Index.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c ProductQuantizer.cpp -o ProductQuantizer.o -DFINTEGER=int
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c AutoTune.cpp -o AutoTune.o
AutoTune.cpp: In member function ‘virtual void faiss::ParameterSpace::initialize(const faiss::Index*)’:
AutoTune.cpp:309:22: warning: unused variable ‘ix’ [-Wunused-variable]
const classname *ix = dynamic_cast<const classname *>(index)
^
AutoTune.cpp:389:9: note: in expansion of macro ‘DC’
if (DC (IndexIVFPQR)) {
^
AutoTune.cpp: In function ‘faiss::Index* faiss::index_factory(int, const char*, faiss::MetricType)’:
AutoTune.cpp:706:40: warning: unused variable ‘pq_m’ [-Wunused-variable]
int d_out, opq_M, nbit, M, M2, pq_m, ncent;
^
AutoTune.cpp:706:46: warning: unused variable ‘ncent’ [-Wunused-variable]
int d_out, opq_M, nbit, M, M2, pq_m, ncent;
^
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c AuxIndexStructures.cpp -o AuxIndexStructures.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c IndexScalarQuantizer.cpp -o IndexScalarQuantizer.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c FaissException.cpp -o FaissException.o
g++ -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -c IndexHNSW.cpp -o IndexHNSW.o -DFINTEGER=int
IndexHNSW.cpp: In member function ‘void faiss::ReconstructFromNeighbors::estimate_code(const float*, faiss::ReconstructFromNeighbors::storage_idx_t, uint8_t*) const’:
IndexHNSW.cpp:1545:13: warning: unused variable ‘d1’ [-Wunused-variable]
int d1 = d0 + dsub;
^
IndexHNSW.cpp: In function ‘int faiss::{anonymous}::search_from_candidates_2(const faiss::HNSW&, faiss::{anonymous}::DistanceComputer&, int, faiss::{anonymous}::idx_t*, float*, faiss::{anonymous}::MinimaxHeap&, faiss::VisitedTable&, int, int)’:
IndexHNSW.cpp:2001:15: warning: unused variable ‘d’ [-Wunused-variable]
float d = candidates.dis[i];
^
ar r libfaiss.a hamming.o utils.o IndexFlat.o IndexIVF.o IndexLSH.o IndexPQ.o IndexIVFPQ.o Clustering.o Heap.o VectorTransform.o index_io.o PolysemousTraining.o MetaIndexes.o Index.o ProductQuantizer.o AutoTune.o AuxIndexStructures.o IndexScalarQuantizer.o FaissException.o IndexHNSW.o
ar: 正在建立 libfaiss.a
g++ -o tests/demo_ivfpq_indexing -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp tests/demo_ivfpq_indexing.cpp libfaiss.a -g -fPIC -fopenmp /usr/lib/libopenblas.so.0
[email protected]:~/faiss$ ./tests/demo_ivfpq_indexing
[0.000 s] Generating 100000 vectors in 128D for training
[0.773 s] Training the index
Training level-1 quantizer
Training level-1 quantizer on 100000 vectors in 128D
Training IVF residual
Input training set too big (max size is 65536), sampling 65536 / 100000 vectors
computing residuals
training 4x256 product quantizer on 65536 vectors in 128D
Training PQ slice 0/4
Clustering 65536 points in 32D to 256 clusters, redo 1 times, 25 iterations
Preprocessing in 0.01 s
Iteration 0 (0.40 s, search 0.37 s): objective=151134 imbalance=2.083 nsplit=0 Iteration 1 (0.83 s, search 0.75 s): objective=116917 imbalance=1.060 nsplit=0 Iteration 2 (1.27 s, search 1.16 s): objective=115683 imbalance=1.017 nsplit=0 Iteration 3 (1.60 s, search 1.46 s): objective=115138 imbalance=1.010 nsplit=0 Iteration 4 (2.16 s, search 2.00 s): objective=114789 imbalance=1.008 nsplit=0 Iteration 5 (2.57 s, search 2.40 s): objective=114542 imbalance=1.007 nsplit=0 Iteration 6 (3.04 s, search 2.86 s): objective=114347 imbalance=1.006 nsplit=0 Iteration 7 (3.50 s, search 3.30 s): objective=114192 imbalance=1.005 nsplit=0 Iteration 8 (3.94 s, search 3.71 s): objective=114064 imbalance=1.005 nsplit=0 Iteration 9 (4.46 s, search 4.21 s): objective=113960 imbalance=1.005 nsplit=0 Iteration 10 (4.97 s, search 4.71 s): objective=113873 imbalance=1.005 nsplit= Iteration 11 (5.34 s, search 5.05 s): objective=113797 imbalance=1.005 nsplit= Iteration 12 (5.86 s, search 5.54 s): objective=113731 imbalance=1.005 nsplit= Iteration 13 (6.40 s, search 6.06 s): objective=113676 imbalance=1.005 nsplit= Iteration 14 (6.90 s, search 6.55 s): objective=113624 imbalance=1.005 nsplit= Iteration 15 (7.41 s, search 7.04 s): objective=113579 imbalance=1.005 nsplit= Iteration 16 (7.96 s, search 7.56 s): objective=113537 imbalance=1.005 nsplit= Iteration 17 (8.35 s, search 7.91 s): objective=113504 imbalance=1.005 nsplit= Iteration 18 (8.81 s, search 8.34 s): objective=113474 imbalance=1.005 nsplit= Iteration 19 (9.37 s, search 8.88 s): objective=113446 imbalance=1.004 nsplit= Iteration 20 (9.85 s, search 9.33 s): objective=113420 imbalance=1.004 nsplit= Iteration 21 (10.35 s, search 9.79 s): objective=113399 imbalance=1.004 nsplit Iteration 22 (10.87 s, search 10.27 s): objective=113378 imbalance=1.004 nspli Iteration 23 (11.26 s, search 10.63 s): objective=113358 imbalance=1.004 nspli Iteration 24 (11.74 s, search 11.09 s): objective=113338 imbalance=1.004 nsplit=0
Training PQ slice 1/4
Clustering 65536 points in 32D to 256 clusters, redo 1 times, 25 iterations
Preprocessing in 0.00 s
Iteration 0 (0.33 s, search 0.30 s): objective=149183 imbalance=2.952 nsplit=0 Iteration 1 (0.69 s, search 0.62 s): objective=116843 imbalance=1.067 nsplit=0 Iteration 2 (1.15 s, search 1.06 s): objective=115476 imbalance=1.016 nsplit=0 Iteration 3 (1.66 s, search 1.55 s): objective=114921 imbalance=1.009 nsplit=0 Iteration 4 (1.99 s, search 1.85 s): objective=114560 imbalance=1.006 nsplit=0 Iteration 5 (2.54 s, search 2.38 s): objective=114312 imbalance=1.006 nsplit=0 Iteration 6 (3.11 s, search 2.94 s): objective=114126 imbalance=1.005 nsplit=0 Iteration 7 (3.58 s, search 3.37 s): objective=113985 imbalance=1.005 nsplit=0 Iteration 8 (4.13 s, search 3.91 s): objective=113872 imbalance=1.005 nsplit=0 Iteration 9 (4.67 s, search 4.43 s): objective=113781 imbalance=1.005 nsplit=0 Iteration 10 (5.20 s, search 4.93 s): objective=113701 imbalance=1.005 nsplit= Iteration 11 (5.74 s, search 5.43 s): objective=113630 imbalance=1.004 nsplit= Iteration 12 (6.17 s, search 5.84 s): objective=113570 imbalance=1.004 nsplit= Iteration 13 (6.70 s, search 6.35 s): objective=113517 imbalance=1.004 nsplit= Iteration 14 (7.05 s, search 6.66 s): objective=113469 imbalance=1.004 nsplit= Iteration 15 (7.57 s, search 7.16 s): objective=113423 imbalance=1.004 nsplit= Iteration 16 (8.04 s, search 7.62 s): objective=113385 imbalance=1.004 nsplit= Iteration 17 (8.55 s, search 8.09 s): objective=113351 imbalance=1.004 nsplit= Iteration 18 (9.05 s, search 8.58 s): objective=113323 imbalance=1.004 nsplit= Iteration 19 (9.56 s, search 9.05 s): objective=113297 imbalance=1.004 nsplit= Iteration 20 (10.08 s, search 9.56 s): objective=113273 imbalance=1.004 nsplit Iteration 21 (10.61 s, search 10.05 s): objective=113248 imbalance=1.004 nspli Iteration 22 (11.05 s, search 10.47 s): objective=113226 imbalance=1.004 nspli Iteration 23 (11.63 s, search 11.02 s): objective=113207 imbalance=1.004 nspli Iteration 24 (12.13 s, search 11.49 s): objective=113188 imbalance=1.004 nsplit=0
Training PQ slice 2/4
Clustering 65536 points in 32D to 256 clusters, redo 1 times, 25 iterations
Preprocessing in 0.00 s
Iteration 0 (0.56 s, search 0.53 s): objective=152236 imbalance=2.501 nsplit=0 Iteration 1 (0.96 s, search 0.89 s): objective=116683 imbalance=1.055 nsplit=0 Iteration 2 (1.53 s, search 1.43 s): objective=115451 imbalance=1.018 nsplit=0 Iteration 3 (2.03 s, search 1.91 s): objective=114919 imbalance=1.012 nsplit=0 Iteration 4 (2.36 s, search 2.22 s): objective=114592 imbalance=1.011 nsplit=0 Iteration 5 (2.90 s, search 2.73 s): objective=114358 imbalance=1.010 nsplit=0 Iteration 6 (3.41 s, search 3.23 s): objective=114184 imbalance=1.009 nsplit=0 Iteration 7 (3.81 s, search 3.61 s): objective=114042 imbalance=1.009 nsplit=0 Iteration 8 (4.31 s, search 4.08 s): objective=113925 imbalance=1.009 nsplit=0 Iteration 9 (4.83 s, search 4.55 s): objective=113826 imbalance=1.009 nsplit=0 Iteration 10 (5.40 s, search 5.09 s): objective=113746 imbalance=1.009 nsplit= Iteration 11 (5.79 s, search 5.45 s): objective=113681 imbalance=1.009 nsplit= Iteration 12 (6.29 s, search 5.91 s): objective=113621 imbalance=1.009 nsplit= Iteration 13 (6.85 s, search 6.44 s): objective=113570 imbalance=1.009 nsplit= Iteration 14 (7.35 s, search 6.93 s): objective=113522 imbalance=1.008 nsplit= Iteration 15 (7.71 s, search 7.25 s): objective=113480 imbalance=1.008 nsplit= Iteration 16 (8.20 s, search 7.72 s): objective=113439 imbalance=1.008 nsplit= Iteration 17 (8.62 s, search 8.13 s): objective=113402 imbalance=1.008 nsplit= Iteration 18 (9.13 s, search 8.62 s): objective=113368 imbalance=1.009 nsplit= Iteration 19 (9.62 s, search 9.07 s): objective=113342 imbalance=1.009 nsplit= Iteration 20 (9.99 s, search 9.41 s): objective=113315 imbalance=1.009 nsplit= Iteration 21 (10.51 s, search 9.91 s): objective=113293 imbalance=1.009 nsplit Iteration 22 (11.01 s, search 10.39 s): objective=113269 imbalance=1.008 nspli Iteration 23 (11.47 s, search 10.84 s): objective=113248 imbalance=1.008 nspli Iteration 24 (12.00 s, search 11.35 s): objective=113227 imbalance=1.009 nsplit=0
Training PQ slice 3/4
Clustering 65536 points in 32D to 256 clusters, redo 1 times, 25 iterations
Preprocessing in 0.00 s
Iteration 0 (0.52 s, search 0.49 s): objective=150530 imbalance=2.238 nsplit=0 Iteration 1 (1.02 s, search 0.96 s): objective=116725 imbalance=1.062 nsplit=0 Iteration 2 (1.54 s, search 1.45 s): objective=115446 imbalance=1.015 nsplit=0 Iteration 3 (2.03 s, search 1.92 s): objective=114878 imbalance=1.008 nsplit=0 Iteration 4 (2.54 s, search 2.41 s): objective=114517 imbalance=1.006 nsplit=0 Iteration 5 (3.06 s, search 2.91 s): objective=114254 imbalance=1.006 nsplit=0 Iteration 6 (3.58 s, search 3.42 s): objective=114056 imbalance=1.005 nsplit=0 Iteration 7 (4.03 s, search 3.83 s): objective=113899 imbalance=1.005 nsplit=0 Iteration 8 (4.49 s, search 4.27 s): objective=113769 imbalance=1.004 nsplit=0 Iteration 9 (5.02 s, search 4.78 s): objective=113664 imbalance=1.004 nsplit=0 Iteration 10 (5.55 s, search 5.30 s): objective=113574 imbalance=1.004 nsplit= Iteration 11 (6.07 s, search 5.79 s): objective=113496 imbalance=1.004 nsplit= Iteration 12 (6.60 s, search 6.29 s): objective=113431 imbalance=1.004 nsplit= Iteration 13 (7.13 s, search 6.80 s): objective=113375 imbalance=1.004 nsplit= Iteration 14 (7.69 s, search 7.33 s): objective=113326 imbalance=1.004 nsplit= Iteration 15 (8.22 s, search 7.83 s): objective=113283 imbalance=1.004 nsplit= Iteration 16 (8.78 s, search 8.37 s): objective=113243 imbalance=1.004 nsplit= Iteration 17 (9.11 s, search 8.67 s): objective=113210 imbalance=1.004 nsplit= Iteration 18 (9.63 s, search 9.16 s): objective=113181 imbalance=1.004 nsplit= Iteration 19 (10.14 s, search 9.65 s): objective=113154 imbalance=1.004 nsplit Iteration 20 (10.66 s, search 10.15 s): objective=113130 imbalance=1.004 nspli Iteration 21 (11.15 s, search 10.61 s): objective=113110 imbalance=1.004 nspli Iteration 22 (11.69 s, search 11.13 s): objective=113093 imbalance=1.004 nspli Iteration 23 (12.06 s, search 11.47 s): objective=113076 imbalance=1.004 nspli Iteration 24 (12.60 s, search 11.98 s): objective=113061 imbalance=1.004 nsplit=0
precomputing IVFPQ tables type 1
[64.371 s] storing the pre-trained index to /tmp/index_trained.faissindex
[64.372 s] Building a dataset of 200000 vectors to index
[64.570 s] Adding the vectors to the index
IndexIVFPQ::add_core_o: adding 0:32768 / 200000
add_core times: 653.988 316.309 2.517
IndexIVFPQ::add_core_o: adding 32768:65536 / 200000
add_core times: 539.748 234.671 1.499
IndexIVFPQ::add_core_o: adding 65536:98304 / 200000
add_core times: 456.647 202.689 1.470
IndexIVFPQ::add_core_o: adding 98304:131072 / 200000
add_core times: 490.088 262.767 1.160
IndexIVFPQ::add_core_o: adding 131072:163840 / 200000
add_core times: 584.106 292.540 1.238
IndexIVFPQ::add_core_o: adding 163840:196608 / 200000
add_core times: 248.710 245.927 1.318
IndexIVFPQ::add_core_o: adding 196608:200000 / 200000
add_core times: 92.542 243.516 0.176
[69.445 s] imbalance factor: 1.23893
[69.449 s] Searching the 5 nearest neighbors of 9 vectors in the index
[69.489 s] Query results (vector ids, then distances):
query 0: 1234 11667 163213 13439 83335
dis: 7.08558 9.57216 9.67245 9.83403 9.83821
query 1: 1235 56663 76478 28798 117320
dis: 7.78506 9.70143 10.1121 10.5115 10.6679
query 2: 1236 80604 189820 185497 81842
dis: 7.52846 11.0073 11.0746 11.1844 11.3107
query 3: 1237 116226 85618 187787 36842
dis: 8.04249 10.4582 10.6636 10.749 10.7599
query 4: 1238 91514 52306 86365 148470
dis: 7.36588 10.1087 10.144 10.2007 10.2791
query 5: 1239 143840 77650 2444 70772
dis: 7.19399 9.78232 9.94378 10.3445 10.4876
query 6: 1240 72767 170354 122756 61561
dis: 7.33832 9.7697 9.9693 10.0013 10.2586
query 7: 1241 30037 162441 194151 196333
dis: 8.14279 11.3589 11.6361 11.6927 11.7009
query 8: 1242 66419 10135 10268 3851
dis: 8.18552 10.7842 11.0947 11.2108 11.2265
note that the nearest neighbor is not at distance 0 due to quantization errors
[email protected]:~/faiss$
[email protected]:~/faiss$ ls
AutoTune.cpp Heap.o INSTALL.md
AutoTune.h Index.cpp libfaiss.a
AutoTune.o IndexFlat.cpp LICENSE
AuxIndexStructures.cpp IndexFlat.h Makefile
AuxIndexStructures.h IndexFlat.o makefile.inc
AuxIndexStructures.o Index.h MetaIndexes.cpp
benchs IndexHNSW.cpp MetaIndexes.h
Clustering.cpp IndexHNSW.h MetaIndexes.o
Clustering.h IndexHNSW.o PATENTS
Clustering.o index_io.cpp PolysemousTraining.cpp
cmake index_io.h PolysemousTraining.h
CMakeLists.txt index_io.o PolysemousTraining.o
CONTRIBUTING.md IndexIVF.cpp ProductQuantizer.cpp
Dockerfile IndexIVF.h ProductQuantizer.h
docs IndexIVF.o ProductQuantizer.o
example_makefiles IndexIVFPQ.cpp python
FaissAssert.h IndexIVFPQ.h README.md
FaissException.cpp IndexIVFPQ.o swigfaiss.swig
FaissException.h IndexLSH.cpp tests
FaissException.o IndexLSH.h tutorial
faiss.h IndexLSH.o utils.cpp
faiss.py Index.o utils.h
gpu IndexPQ.cpp utils.o
hamming.cpp IndexPQ.h VectorTransform.cpp
hamming.h IndexPQ.o VectorTransform.h
hamming.o IndexScalarQuantizer.cpp VectorTransform.o
Heap.cpp IndexScalarQuantizer.h
Heap.h IndexScalarQuantizer.o
[email protected]:~/faiss$ wget ftp://ftp.irisa.fr/local/texmex/corpus/sift.tar.gz
--2018-02-08 14:32:13-- ftp://ftp.irisa.fr/local/texmex/corpus/sift.tar.gz
=> “sift.tar.gz”
正在解析主機 ftp.irisa.fr (ftp.irisa.fr)... 131.254.254.45, 2001:660:7303:254::45
正在連線 ftp.irisa.fr (ftp.irisa.fr)|131.254.254.45|:21... 已連線。
正在以 anonymous 登入 ... 登入成功！
==> SYST ... 完成。 ==> PWD ... 完成。
==> TYPE I ... 完成。 ==> CWD (1) /local/texmex/corpus ... 完成。
==> SIZE sift.tar.gz ... 168280445
==> PASV ... 完成。 ==> RETR sift.tar.gz ... 完成。
長度：168280445 (160M) (非正式資料)
sift.tar.gz 100%[===================>] 160.48M 10.1MB/s in 17s
2018-02-08 14:32:34 (9.21 MB/s) - “sift.tar.gz” 已儲存 [168280445]
[email protected]:~/faiss$
[email protected]:~/faiss$ tar -xzvf sift.tar.gz
sift/
sift/sift_base.fvecs
sift/sift_groundtruth.ivecs
sift/sift_learn.fvecs
sift/sift_query.fvecs
[email protected]:~/faiss$
[email protected]:~/faiss$ mv sift sift1M
[email protected]:~/faiss$
[email protected]:~/faiss$ make tests/demo_sift1M
g++ -o tests/demo_sift1M -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp tests/demo_sift1M.cpp libfaiss.a -g -fPIC -fopenmp /usr/lib/libopenblas.so.0
tests/demo_sift1M.cpp: In function ‘float* fvecs_read(const char*, size_t*, size_t*)’:
tests/demo_sift1M.cpp:50:33: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result [-Wunused-result]
fread(&d, 1, sizeof(int), f);
^
[email protected]:~/faiss$
[email protected]:~/faiss$ ./tests/demo_sift1M
[0.000 s] Loading train set
[0.043 s] Preparing index "IVF4096,Flat" d=128
[0.043 s] Training on 100000 vectors
WARNING clustering 100000 points to 4096 centroids: please provide at least 159744 training points
[24.819 s] Loading database
[25.063 s] Indexing database, size 1000000*128
[50.138 s] Loading queries
[50.139 s] Loading ground truth for 10000 queries
[50.143 s] Preparing auto-tune criterion 1-recall at 1 criterion, with k=100 nq=10000
[50.146 s] Preparing auto-tune parameters
[50.146 s] Auto-tuning over 1 parameters (12 combinations)
0/12: cno=0 nprobe=1 bounds [perf<=1.000 t>=0.000] perf 0.428 t 0.407 *
1/12: cno=11 nprobe=2048 bounds [perf<=1.000 t>=0.407] perf 0.992 t 66.673 *
2/12: cno=4 nprobe=16 bounds [perf<=0.992 t>=0.407] perf 0.887 t 1.267 *
3/12: cno=8 nprobe=256 bounds [perf<=0.992 t>=1.267] perf 0.992 t 10.369 *
4/12: cno=1 nprobe=2 bounds [perf<=0.887 t>=0.407] perf 0.560 t 0.553 *
5/12: cno=5 nprobe=32 bounds [perf<=0.992 t>=1.267] perf 0.948 t 1.859 *
6/12: cno=2 nprobe=4 bounds [perf<=0.887 t>=0.553] perf 0.685 t 0.709 *
7/12: cno=6 nprobe=64 bounds [perf<=0.992 t>=1.859] perf 0.976 t 3.167 *
8/12: cno=3 nprobe=8 bounds [perf<=0.887 t>=0.709] perf 0.799 t 0.891 *
9/12: cno=9 nprobe=512 bounds [perf<=0.992 t>=10.369] perf 0.993 t 19.099 *
10/12: cno=10 nprobe=1024 bounds [perf<=0.992 t>=19.099] skip
11/12: cno=7 nprobe=128 bounds [perf<=0.992 t>=3.167] perf 0.988 t 5.747 *
[160.904 s] Found the following operating points:
Tested 11 operating points, 11 ones are optimal:
cno=-1 key= perf=0.0000 t=0.000
cno=0 key=nprobe=1 perf=0.4285 t=0.407
cno=1 key=nprobe=2 perf=0.5603 t=0.553
cno=2 key=nprobe=4 perf=0.6851 t=0.709
cno=3 key=nprobe=8 perf=0.7993 t=0.891
cno=4 key=nprobe=16 perf=0.8869 t=1.267
cno=5 key=nprobe=32 perf=0.9478 t=1.859
cno=6 key=nprobe=64 perf=0.9764 t=3.167
cno=7 key=nprobe=128 perf=0.9876 t=5.747
cno=8 key=nprobe=256 perf=0.9918 t=10.369
cno=9 key=nprobe=512 perf=0.9927 t=19.099
[160.906 s] Setting parameter configuration "nprobe=2" on index
[160.906 s] Perform a search on 10000 queries
[161.445 s] Compute recalls
[email protected] = 0.5603
[email protected] = 0.5636
[email protected] = 0.5636
[email protected]:~/faiss$
[email protected]:~/faiss$ ./tests/test_blas^C
[email protected]:~/faiss$ make py
python -c++ -Doverride= -o python/swigfaiss_wrap.cxx swigfaiss.swig
File "<string>", line 1
++
^
SyntaxError: invalid syntax
Makefile:79: recipe for target 'python/swigfaiss_wrap.cxx' failed
make: [python/swigfaiss_wrap.cxx] Error 1 (ignored)
g++ -I. -fPIC -m64 -Wall -g -O3 -mavx -msse4 -mpopcnt -fopenmp -Wno-sign-compare -std=c++11 -fopenmp -g -fPIC -fopenmp -I/home/JIAO/anaconda2/include/python2.7/ -I/home/JIAO/anaconda2/lib/python2.7/site-packages/numpy/core/include/ -shared \
-o python/_swigfaiss.so python/swigfaiss_wrap.cxx libfaiss.a /usr/lib/libopenblas.so.0
cp python/_swigfaiss.so python/swigfaiss.py .
[email protected]:~/faiss$ make py
python -c++ -Doverride= -o python/swigfaiss_wrap.cxx swigfaiss.swig
File "<string>", line 1
++
^
SyntaxError: invalid syntax
Makefile:79: recipe for target 'python/swigfaiss_wrap.cxx' failed
make: [python/swigfaiss_wrap.cxx] Error 1 (ignored)
[email protected]:~/faiss$ make py
python -c++ -Doverride= -o python/swigfaiss_wrap.cxx swigfaiss.swig
File "<string>", line 1
++
^
SyntaxError: invalid syntax
Makefile:79: recipe for target 'python/swigfaiss_wrap.cxx' failed
make: [python/swigfaiss_wrap.cxx] Error 1 (ignored)
相關推薦
Ubuntu 16.04 下編譯安裝Faiss CPU版本
環境：unbuntu 16.04安裝anaconda2#下載
wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda2-4.3.0-Linux-x86_64.sh
#許可權
chmod +x An
RabbitMQ在Ubuntu 16.04下的安裝與配置
Linux安裝RabbitMQ
由於rabbitMq需要erlang語言的支持，在安裝rabbitMq之前需要安裝erlang，執行命令：
sudo apt-get install erlang-nox安裝rabbitMq命令：
2.$ sudo apt-get update3.$ sudo apt-get
[環境配置]Ubuntu 16.04 原始碼編譯安裝OpenCV-3.2.0+OpenCV_contrib-3.2.0及產生的問題
1.OpenCV-3.2.0+OpenCV_contrib-3.2.0編譯安裝過程
1）下載官方要求的依賴包
GCC 4.4.x or later
CMake 2.6 or higher
Git
GTK+2.x or higher, including headers (libgtk2.
Ubuntu 16.04下Spark安裝與開發環境搭建
https://blog.csdn.net/lengconglin/article/details/77847623
一、Spark執行環境 Spark是Scala寫的，執行在JVM上，所以執行環境Java7+，如果使用Python
記一次在64位Ubuntu 16.04下成功安裝arm-none-eabi-gcc交叉編譯器的過程
2018.07.28
剛開始在網上找了很多教程，都是大同小異的步驟：
在官網下載arm-none-eabi-gcc的.tar.bz2壓縮包
解壓到自定義目錄
開啟.bashrc和.profie新增環境變數和路徑
生效更改
查詢編譯器是否配置成功
上面的安
ubuntu 16.04下編譯Qt5.9.1
1.下載Qt原始碼：qt-everywhere-opensource-src-5.9.1.tar.xz
2.解壓 tar xvJf qt-everywhere-opensource-src-5.9.1.tar.xz
3.切換到該目錄：
./configure -sta
ubuntu 12.04下編譯安裝nginx-1.9.3之後 tomcat叢集
一、tomcat安裝
1.安裝nginx 請檢視http://blog.csdn.net/liangzi4454/article/details/47023447
2. jdk安裝不在贅述，不會的同學，可以網上查詢教程；
3. tomcat下載
Ubuntu 14.04下編譯安裝Vim7.4
原文連線：http://www.linuxidc.com/Linux/2016-04/129774.htm
1、下載Vim74和適合7.4版本的vim-gdb：
如果已經通過軟體中心安裝了vim，則需要將其解除安裝
sudo apt-get r
RabbitMQ在Ubuntu 16.04下的安裝與配置及python RabbitMQ佇列使用
一、RabbitMQ在Ubuntu 16.04下的安裝與配置新增源echo 'deb http://www.rabbitmq.com/debian/ testing main' | sudo tee /etc/apt/sources.list.d/rabbitmq.list新
Ubuntu 16.04原始碼編譯安裝PHP 5.6.29
2、解壓
tar -zxvf php-5.6.29.tar.gz
3、配置
./configure –prefix=/usr/local/php –with-apxs2=/u
在ubuntu 16.04下編譯android和linux版webrtc
checkout rec ppr 這一 腳本 -- pytho 新源 alex 1. 安裝幹凈的ubuntu 16.04 x86_64位版本
2. 使用apt-get安裝git和Python2.7
3. 下載depot_tools(https://storage.goo
Ubuntu 16.04下CPU only模式的caffe安裝
本文的內容主要參考了Ubuntu16.04安裝Caffe(CPU Only)的工作，該篇博文比較符合我的需要，網上很多的caffe安裝的教程都是帶了GPU的，對我現在而言，不需要，該篇博文較好的滿足我的需要。同時按照該篇博文進行安裝過程的工作中，發現了若干問題，完全按照該部
在Ubuntu 16.04下安裝nodejs
.cn ges ubuntu 技術 com apt-get 分享 6.0 images 源安裝:
1.curl -sL https://deb.nodesource.com/setup_5.x | sudo -E bash -
2.sudo apt-get insta
Ubuntu 16.04下使用VirtualBox虛擬機安裝Windows XP win7 linux 安裝虛擬機
tle .aspx 定義 支持 ins amd 幫助 不用 1.2 1.
http://www.linuxidc.com/Linux/2016-09/135192.htm
2.
對於初次使用Ubuntu 16.04的多小夥伴來說，可能還不是很熟悉操作，有時候，我們就需要在這
Ubuntu 16.04下安裝Apache壓力測試工具ab
apt-get gpo 簡單使用 utils sta markdown 測試結果 壓力測試 安裝apache 安裝
sudo apt-get install apache2-utils
簡單使用
# 對http://www.baidu.com/進行100次請求，10個並發請
Ubuntu 16.04下使用VMware安裝CentOS 6.9然後在裏面再安裝KVM之後配置網橋無法上網的問題
div 使用 ubuntu inux 成功 log 無法上網 ubun centos 6 別試了，這個問題無法解決，已經使用VMware安裝了一層虛擬機，然後在裏面再安裝KVM，網絡層次已經變了，即使配置了網橋同樣無法上網。
可是這樣試一下，不一定成功：把VMware的虛
Ubuntu 16.04下為Android編譯OpenCV 3.2.0 Manager
由於 ica 如何 setting -i add 標記 roi stripe http://johnhany.net/2016/07/build-opencv-manager-for-android-on-ubuntu/
最近想在Android上嘗試一下SIFT和SU
ubuntu 16.04下的fastadmin安裝指南
curl access pan col tps tag tab efault systemctl 此篇博客轉載於fastadmin論壇,方便自己看轉到了博客裏
說明文檔不多，特制作一個，方便大家交流使用Ubuntu 16.04 安裝fastadmin指南本文因考慮到大多數人
Ubuntu 16.04下OLSR協議安裝教程
密碼 border 旗艦版 eas war alt 語法分析器 獲取 同傳
OLSR是根據MANET的要求，在傳統的LS（Link state）協議的基礎上優化的。
OLSR中的關鍵概念是多點轉播（MPRs），MPRs是在廣播洪泛的過程中挑選的轉發廣播的節點。傳統
Ubuntu 16.04 下安裝nginx
一、建立一個目錄存放nginx認證祕鑰。命令列進入該目錄 執行 wget http://nginx.org/keys/nginx_signing.key 會在當前目錄中下載該網頁檔案 二、執行sudo apt-key add nginx_signing.key命令將nginx金鑰加